# File Upload with S3 Flow

This document describes the file upload pipeline using **multer** (multipart parsing) and **AWS S3** (cloud storage) in the `cms-service`. The same pattern is used across `admin-service`, `academic-core-service`, `academic-operation-service`, and `admin-master-service`.

---

## Architecture Overview

```
Client (multipart/form-data)
  │
  ▼
Express Router
  │
  ▼
auth(["admin"]) middleware  ── JWT verification (@dk/shared)
  │
  ▼
handleMulterErrors(upload.fields([...]))  ── Parses files into req.files (memory)
  │
  ▼
Validation middleware  ── express-validator
  │
  ▼
Controller
  ├── uploadToS3(files, "Folder/Subfolder", true)  ── Uploads to S3
  ├── Creates/Updates MongoDB document with S3 URLs
  └── On error → removeImageFromS3() rollback
```

---

## Environment Variables

Define these in your service's `.env` file (e.g. `cms-service/.env`):

| Variable | Example Value | Description |
|---|---|---|
| `AWS_ACCESS_KEY_ID` | `AKIAUDC3E6GKVISJXSTT` | AWS IAM access key |
| `AWS_SECRET_ACCESS_KEY` | `rA7pEzpL6tzKNiiZNHCMYaPypvduyN0EbEjbEUL1` | AWS IAM secret key |
| `AWS_REGION` | `ap-southeast-1` | AWS region (default: `ap-southeast-1`) |
| `BUCKET_NAME` | `diamond-knowledge` | S3 bucket name |

---

## Configuration Files

### 1. S3 Client — `config/s3Config.js`

Creates and exports a singleton `S3Client` instance:

```js
const { S3Client } = require("@aws-sdk/client-s3");
require("dotenv").config();

const s3 = new S3Client({
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  },
  region: process.env.AWS_REGION || "ap-southeast-1",
});

module.exports = { s3 };
```

### 2. File Uploader — `config/fileUploader.js`

Central module that exports four utilities:

#### `upload` (multer instance)

- **Storage**: `memoryStorage()` — files held in memory as `Buffer`
- **Size limit**: 10 MB
- **Allowed extensions**:
  Images: `.png`, `.avif`, `.webp`, `.jpg`, `.jpeg`, `.gif`
  Video/Audio: `.mp4`, `.mp3`, `.mov`
  Documents: `.pdf`, `.doc`, `.docx`, `.xls`, `.xlsx`, `.ppt`, `.pptx`, `.txt`, `.csv`
  Other: `.zip`
- Invalid file types produce `new Error("Invalid file type")`

#### `handleMulterErrors(multerUpload)`

Wraps a multer middleware (e.g. `upload.fields(...)`) with error handling:

| Error | HTTP Status | Body |
|---|---|---|
| `LIMIT_FILE_SIZE` | `413` | `{ error: "File too large", message: "Maximum file size is 10MB" }` |
| `LIMIT_UNEXPECTED_FILE` | `400` | `{ error: "Invalid field", message: "Unexpected field in form data" }` |
| Other multer errors | `400` | `{ error: "Upload error", message: <err.message> }` |

#### `uploadToS3(files, folderPath, isLocationOnly)`

- **files**: single file or array of files (from `req.files`)
- **folderPath**: S3 key prefix (e.g. `"SocialService/Images"`)
- **isLocationOnly**: if `true`, returns only the S3 URL string; otherwise returns `{ location, key, ...metadata }`
- Constructs S3 key as: `folderPath/Date_timestamp_originalFilename`
- Uses `@aws-sdk/lib-storage` `Upload` class (supports multipart for large files)
- Strips `buffer` from returned metadata

#### `removeImageFromS3(imageURL)`

- Extracts the S3 key from the full URL
- Sends `DeleteObjectCommand` to remove the object
- Returns `{ success: true, message: "Image removed from S3" }`

---

## Route Integration Pattern

Reference: `cms-service/src/routes/socialService.routes.js`

```js
const { handleMulterErrors, upload } = require("../../config/fileUploader");

router.route("/social-service").post(
  auth(["admin"]),
  handleMulterErrors(
    upload.fields([
      { name: "images", maxCount: 10 },
      { name: "organizerImage", maxCount: 1 },
    ]),
  ),
  validate(socialServiceValidation.createSocialServiceValidator),
  socialServiceController.createSocialService,
);
```

### Field Configuration

Each field in `upload.fields()` specifies:
- **`name`** — must match the form-data field name from the client
- **`maxCount`** — maximum number of files allowed for that field

---

## S3 Folder Structure

Files are organized under the shared bucket (`diamond-knowledge`) by service and entity:

```
diamond-knowledge/
├── SocialService/
│   ├── Images/
│   │   └── 1718000000_photo.jpg
│   └── OrganizerImage/
│       └── 1718000000_headshot.png
├── Product/
│   └── ...
└── ...
```

---

## Controller Usage Pattern

Reference: `cms-service/src/controllers/socialService.controller.js`

### Create (with rollback)

```js
const create = async (req, res) => {
  const uploadedImages = [];
  try {
    const files = req.files;

    if (files.images) {
      const images = await uploadToS3(files.images, "SocialService/Images", true);
      data.images = images;
      uploadedImages.push(...images);
    }

    if (files.organizerImage) {
      const organizerImage = await uploadToS3(
        files.organizerImage, "SocialService/OrganizerImage", true,
      );
      data.organizer.imageUrl = organizerImage[0];
      uploadedImages.push(...organizerImage);
    }

    const doc = await Model.create(data);
    // success response...
  } catch (error) {
    // Rollback: delete all uploaded images on failure
    for (const image of uploadedImages) {
      await removeImageFromS3(image);
    }
    return InternalServerError(res, error);
  }
};
```

### Update (remove old + upload new)

1. Parse `removeImagesIndex` and `removeOrganizerImage` from validated body
2. Collect old image URLs into `toRemovedImages`
3. Upload new images via `uploadToS3`
4. Update the MongoDB document
5. Delete old images from S3 via `removeImageFromS3`
6. On error — rollback newly uploaded images

### Delete (cleanup)

1. Fetch existing document
2. Delete all associated S3 images via `removeImageFromS3`
3. Delete the MongoDB document

---

## Key Points

- **Memory storage**: Files are kept in RAM as buffers — suitable for moderate file sizes; for very large files, consider switching to disk storage.
- **isLocationOnly=true**: When only the URL string is needed (stored in DB), pass `true` to get a flat string array instead of objects.
- **Rollback pattern**: Always track uploaded image URLs and delete them on failure to avoid orphaned S3 objects.
- **Allowed extensions**: Centralized in the `ALLOWED_EXTENSIONS` Set — extend it as new file types are needed.
- **Auth**: File upload routes are protected by `auth(["admin"])` — ensure the JWT secret (`JWT_SECRET`) matches across services.

---

## Dependencies (package.json)

```json
"@aws-sdk/client-s3": "^3.914.0",
"@aws-sdk/lib-storage": "^3.914.0",
"multer": "^2.0.2",
"dotenv": "^17.2.3"
```