Files & Uploads — the FileUpload storage subsystem

The whole file subsystem reduces to one idea: Every uploaded file is one FileUpload row whose uri points at an object in S3. The binary lives in the bucket; the row is the catalogue entry. Other entities never embed bytes — they store the FileUpload._id (or query by refId) and resolve the row on read.

Source: BE src/modules/upload + library zync-nest-library/upload (UploadService, BucketService, ApUploadUrlScalar) · Admin src/modules/fileUpload

Naming note. This is the file upload subsystem (the upload BE module, GraphQL type FileUpload). It is not a fixed-asset register — that is the separate Assets domain (../domains/assets/fixed-assets-depreciation.md), which merely consumes this subsystem to attach images: [FileUpload!].

1. Purpose & scope

Responsible for:

• Accepting binary file uploads over GraphQL (multipart Upload scalar), streaming them to object storage (S3-compatible bucket), and recording one FileUpload document per stored object.
• Serving files back as absolute URLs via the ApUploadUrl custom scalar (prefixes the stored bucket key with the public CDN/bucket base URL on serialize).
• Deleting both the catalogue row and the underlying object.
• Optional image resizing into sm/md/lg variants (via sharp) — exposed by the library but not wired through the zerp uploadFile mutation (no sizes passed; see §9).

Explicitly does NOT:

• Generate pre-signed/time-limited URLs. Objects are uploaded with ACL public-read and served by a plain public URL. There is no per-request signing (§4, §9).
• Enforce file-type or per-file size limits at the application layer beyond the global multipart guard (maxFiles: 25, maxFileSize: 100 MB) set in main.ts (§4).
• Own the consumer relationships — each domain (KYC, orders, assets, recruitment, training, company) decides how it links to FileUpload rows (§6, §8).

2. Data model

2.1 FileUpload — the file catalogue (collection file_uploads)

One row per stored object. Soft-deletable; tenant/branch scoped via BaseSchema (companyId, branchId). The binary itself is not in Mongo — only the uri key.

// upload/upload.schema.ts
@ApSchema({ collection: "file_uploads", timestamps: true })
export class FileUpload extends BaseSchema {
  @Prop({ unique: true }) ref: string;   // generated document number (unique)
  @Prop() uri: string;                    // bucket key — "<bucket>/<baseKey>/<uuid>.<ext>"
  @Prop() name: string;                   // original filename (file.filename)
  @Prop() cover: boolean;                 // flag: is this the cover/primary image?
  @Prop() type: string;                   // MIME type (file.mimetype) e.g. "image/png"
  @Prop() module: string;                 // origin module tag: "upload" | "kyc" | "order" | ...
}
FileUploadSchema.plugin(SoftDelete, { deletedAt: true });

refId is on the DTO, not persisted as a schema @Prop. FileUpload (the GraphQL ObjectType in upload/upload.dto.ts) declares refId: string | string[] (scalar ApStringOrArray), and the repository's buildQuery() casts it to ObjectId(s) for $in filtering — yet the persisted FileUpload class (upload.schema.ts) has no refId prop. In practice consumers that use findByRefId() (e.g. recruitment attachments) pass the owner's _id as refId at upload time via the IFileUpload.refId field; it is stored on the spread document but is untyped. Treat refId as a soft convention, not a guaranteed indexed column. The robust linking pattern is the owner storing FileUpload._id (KYC, company, order all do this — §6).

There are no enums on this schema. module/type are free-text strings.

2.2 GraphQL shape (upload/upload.dto.ts → schema.gql)

type FileUpload {
  _id: String
  ref: String
  refId: ApStringOrArray
  uri: ApUploadUrl        # ← custom scalar: serializes the bucket key to an absolute URL
  name: String
  cover: Boolean
  type: String
  module: String
  companyId: String  branchId: String
  createdBy: String  createdAt: Float  updatedAt: Float
  canDelete: Boolean
}

scalar Upload          # graphql-upload-ts multipart scalar (request side)
scalar ApUploadUrl     # zerp custom scalar (response side, §4.3)

input FileUploadInput { file: Upload  files: [Upload!] }
input FileUploadPageInput { skip: Float!  take: Float!  keyword: String }
type FileUploadPageResult { totalRecords: Float!  data: [FileUpload!]! }

3. API surface

Resolver: upload/upload.resolver.ts — FileUploadResolver extends ApBaseResolver<FileUpload>, guarded by @ApGqlAuthorize() and audited via @AuditMeta({ module:'upload', collection:'file_uploads' }) (snapshots on CREATE and DELETE → audit trail).

@Mutation((returns) => [FileUpload], { name: "uploadFile" })
public async upload(@GqlCurrentUser() user, @Args("refId") refId, @Args("file") file: FileUploadInput) {
  const fl = await file.file;                 // resolve the multipart promise
  if (fl) return this.fileUploadSvc.upload({  // → §4.1
    files: [fl], type: "stream", module: "upload", refId,
  });
  throw new BadRequestException("No file found in the request");
}

The uploadFile mutation accepts only a single file (it reads file.file, ignores file.files). The plural files: [Upload!] exists on the input but the resolver does not iterate it. Multi-file uploads in zerp go through domain-specific mutations (uploadSalesReceipt, uploadPurchaseReceipt — §6.2) that pass arrays straight to fileUploadSvc.upload({ files }).

REST: FileUploadController (@Controller("api/file-uploads")) exists but is empty — no routes. All file traffic is GraphQL. Serving of the bytes is the bucket's own HTTP endpoint, not a NestJS route.

4. Upload & storage mechanics

The flow has three layers: zerp module (FileUploadService) → library (UploadService) → bucket (BucketService, the actual S3 client).

4.1 FileUploadService.upload() — zerp orchestration (upload/upload.service.ts)

public async upload(upload: IFileUpload): Promise<FileUpload[]> {
  const uploads = await this.uploadSvc.saveFiles(upload.files, upload.sizes, upload.type); // → S3
  const savedFiles: FileUpload[] = [];
  for await (const up of uploads) {
    savedFiles.push(await this.fileUploadRepo.create({
      ...upload, ...up,                          // up = { name, type, uri } from S3
      createdBy: this.contextSvc?.user?._id,
    }));
  }
  return savedFiles;                             // one FileUpload row per stored object
}

IFileUpload (upload/upload.interface.ts): { files, sizes?, type: "stream"|"base64", module, refId? }. Storage happens first, then the row is written with the returned uri. (No transaction wraps the two; an orphaned object is possible if the Mongo write fails after the S3 put — §9.)

4.2 UploadService.saveFiles() — library, MIME + resize (zync-nest-library/upload/upload.service.ts)

async saveFiles(files, sizes?, type = "base64"): Promise<IUploadResult[]> {
  for await (let file of files) {
    const mapped = await this._mapFile(file, type);   // stream: await { createReadStream, filename, mimetype }
    const uri    = await this._saveFile(mapped, type);// → bucketSvc.uploadStream | uploadBase64
    const fl = { name: file.filename, type: file.mimetype, uri };
    if (sizes?.length) {                              // OPTIONAL image variants (not used by zerp uploadFile)
      // resize via sharp to SIZE_MAP[size] and store each → fl.smUri / fl.mdUri / fl.lgUri
    }
  }
}
const SIZE_MAP = { sm: 433, md: 640, lg: 800 };       // longest-edge px, sharp fit:"inside"

• type: "stream" (what zerp uses): awaits the graphql-upload promise, takes createReadStream(), pipes the stream to S3.
• type: "base64": decodes a data-URI string to a Buffer first.
• Resize uses sharp(...).resize({ height, width, fit: inside, withoutEnlargement: true }) and stores each variant under a size-named sub-dir (433/, 640/, 800/). Only triggered when sizes is passed — which the zerp uploadFile path never does.

4.3 BucketService — the S3 client (zync-nest-library/upload/bucket/bucket.service.ts)

S3-compatible storage via aws-sdk v2. Config comes from the bucket config namespace (bucket.config.ts), populated from env:

new AWS.S3({
  endpoint: config.endpoint,            // aws_s3_endpoint
  sslEnabled: false,                    // ← plain HTTP to the endpoint
  s3ForcePathStyle: true,               // path-style (endpoint/bucket/key) — DigitalOcean/MinIO style
  credentials: { accessKeyId, secretAccessKey },
  region: config.region,
  params: { ACL: config.acl, Bucket: config.bucket },  // ACL = "public-read"
});

Upload (_processUpload) — the only writer:

const response = await this.s3.upload({
  Body: upload.file,
  Key: dir ? `${baseKey}/${dir}/${upload.filename}` : `${baseKey}/${upload.filename}`,
  ContentType: upload.filetype,
}).promise();
return `${response.Bucket}/${response.Key}`;   // stored as FileUpload.uri

• Filename is rewritten to a UUID before storage (uuidFilenameTransform(filename) → <uuidv4><ext>) unless disableTransformName is set. Original name is preserved separately in FileUpload.name. This prevents collisions and path traversal.
• Stored uri = "<bucket>/<baseKey>/<uuid>.<ext>" (path-style, includes the bucket name).

Delete (deleteFile(uri)) — keys off the last path segment of the stored uri and rebuilds ${baseKey}/${lastSegment}, then s3.deleteObject. (Note: it does not await the .promise(), and ignores dir sub-folders — size variants and any dir-nested objects are not removed; §9.)

4.4 Serving — ApUploadUrlScalar (zync-nest-library/upload/upload.scalar.ts)

The uri field is typed ApUploadUrl. On serialize (response), the stored bucket key is turned into an absolute URL; values already absolute are passed through:

serialize(value) {
  return value?.includes("http") ? value : `${process.env.aws_s3_url}/${value}`;
}

So a stored uri of zerp-bucket/development/abc.png is returned to the client as <aws_s3_url>/zerp-bucket/development/abc.png. No signing, no expiry — the URL is permanent and public (matches the public-read ACL). This is the entire "serving" mechanism; the browser fetches the object directly from the bucket/CDN.

4.5 Request-side multipart limits (src/main.ts)

app.use(graphqlUploadExpress({ maxFiles: 25, maxFileSize: 100 * 1024 * 1024, maxFieldSize: 100 * 1024 * 1024 }));

Global guard via graphql-upload-ts: max 25 files / 100 MB per file / 100 MB per field per request. There is no per-MIME-type allow-list in code — any file type is accepted (type is recorded, not validated). See architecture for where this sits in bootstrap.

5. Permissions

• FileUploadResolver is class-decorated @ApGqlAuthorize() → a valid JWT is required for all three operations (see auth). There is no dedicated permission module/action gate (@ApGqlAuthorize() is used without a module/action arg, so it's authentication-only, not RBAC).
• Tenant scope is enforced implicitly: every row carries companyId/branchId from BaseSchema, and reads go through the repository which scopes by tenant context (see permissions-access and multi-tenancy).
• Consumer mutations (uploadSalesReceipt, updateKycDoc, etc.) carry their own module's @AuditMeta and whatever authorization their resolver declares — file access rides on the owning domain's permissions.

6. How other entities reference files

There are three linking conventions in zerp. All store IDs, never bytes.

6.1 Single-file FK (<thing>Id → resolve to FileUpload)

The owning document stores a FileUpload._id; a @ResolveField hydrates it on read.

• Company — logoId, letterHeadHeaderId, letterHeadFooterId (company/company.resolver.ts): logo() → fileUploadSvc.findById(args.logoId).
• KYC — idFrontId, idBackId, selfieId (kyc/kyc.resolver.ts). The updateKycDoc mutation uploads each provided image with module: "kyc", stores the returned _id, and rolls back (deletes) the uploaded files if the KYC create fails — the closest thing to transactional upload in the codebase. See ../domains/crm/kyc.md.

// kyc.resolver.ts — upload-then-link, with compensating delete on failure
const idFront = kyc.idFront ? (await this.fileUploadSvc.upload({ files:[kyc.idFront], type:"stream", module:"kyc" }))[0] : null;
if (idFront?._id) (kyc as any).idFrontId = idFront._id;
return this.KYCSvc.create({ ...kyc }).catch(err => { if (idFront) this.fileUploadSvc.delete(idFront._id) /* ...others */ });

6.2 Array of FKs (receiptIds: [String] → receipts: [FileUpload])

The owner stores an array of FileUpload._id; a resolve-field maps each to a row.

• Orders / Sales / Purchase / Finance transactions / Assets all expose receipts: [FileUpload!] (orders/transactions) or images: [FileUpload!] (assets).
• Upload entry points: uploadSalesReceipt / uploadPurchaseReceipt (OrderReceiptInput { salesId, files: [Upload] }) → fileUploadSvc.upload({ files, type:"stream", module:"order" }) → store receiptIds = files.map(f => f._id) on the order.

// purchase.resolver.ts → uploadPurchaseReceipt
const files = await this.fileUploadSvc.upload({ files: receipt.files, type: "stream", module: "order" });
await this.purchaseSvc.update(receipt.salesId, { receiptIds: files.map(f => f._id) });
// resolve-field: receipts() → Promise.all(args.receiptIds.map(r => fileUploadSvc.findById(r)))

Consumers: ../domains/inventory/purchases.md, ../domains/inventory/sales.md, ../domains/sales-pos/order-flows.md, ../domains/finance/transaction.md, ../domains/assets/fixed-assets-depreciation.md.

6.3 Reverse lookup by refId (findByRefId)

The owner stores nothing; files carry the owner's id as refId, and a resolve-field queries fileUploadSvc.findByRefId(owner._id).

• Recruitment job applicant — attachments: [FileUpload!] resolves via this.fileUploadSvc.findByRefId(applicant._id) (recruitment/job-applicant/job-applicant.resolver.ts). See ../domains/recruitment/recruitment.md.

6.4 Server-generated files (no client upload)

• HR training certificate — TrainingCertificateService.issuePdfCertificate() renders a PDF with pdfkit into a Buffer, then calls uploadSvc.upload.uploadStream({ file: pdfBuffer, ... }) directly on the bucket (bypassing FileUploadService/FileUpload rows) and stores the returned uri string in TrainingCertificate.certificateUrl. So a training cert URL is a raw bucket key string, not a FileUpload reference. See ../domains/hr/training.md.

7. Admin UI

Admin module src/modules/fileUpload is a thin, headless wrapper (no page of its own — it's a shared context other feature screens consume).

• context.tsx — UploadFileContextProvider exposing useUploadFileState() with:
  • uploadFile(refId, file) → useUploadFileQuery().uploadFile({ variables: { refId, file: { files: file } } })
  • deleteFile(id) → deleteFile({ variables: { id } })
  • loading, files state. (Follows the zync-nextjs context-owns-state rule: components call useUploadFileState(), never the gql directly.)
• gql/query.ts — UPLOAD_FILE (uploadFile(refId, file)) and DELETE_FILE (deleteFile(_id)) mutations; FileUploadFragment selects _id ref key refId uri name cover type module createdBy createdAt updatedAt canDelete.
• model.ts — IFileUpload, IFileUploadInput { file?, files? }, IFileUploadQuery.

Known bug in admin context: uploadFile() reads res?.data?.updateAsset — a leftover from an asset screen — instead of res?.data?.uploadFile. As written it will not surface the returned rows for generic use; feature screens that need the result generally call the mutation through their own module. Flag when porting (§9).

Image rendering uses the absolute uri returned by ApUploadUrl directly in <img>/avatar components — no client-side signing.

8. Dependencies & integrations

• zync-nest-library → ApUploadModule provides UploadService + ApUploadUrlScalar; under it, BucketModule provides BucketService (the S3 client). FileUploadModule imports ApUploadModule and AuthModule, registers the FileUpload Mongoose model, and exports FileUploadService so every consumer module can inject it (upload/upload.module.ts).
• External service: S3-compatible object storage (aws-sdk v2, path-style, plain HTTP, public-read ACL). Sanity-checked by the repo-root s3-test.js (s3.putObject smoke test using the same env vars).
• sharp — image resize (variant generation; library-side, optional).
• graphql-upload-ts — multipart Upload scalar + graphqlUploadExpress middleware.
• pdfkit — server-side PDF generation for training certificates (§6.4).
• Consumers (inject FileUploadService): company, KYC, recruitment/job-applicant, inventory order/sales/purchase, inventory item, finance transaction/payment, assets, workflow event/task, user.
• Audit: uploadFile / deleteFile snapshot to the audit trail.
• No cron/jobs and no orphan-GC process exist for the bucket.

9. Gotchas & project-specific rules

1. Public URLs, no signing/expiry. Objects are public-read; ApUploadUrl just prefixes aws_s3_url. Anyone with the URL can fetch the file forever. There is no access check on serve and no pre-signed-URL path. If a private/expiring scheme is needed it must be added.
2. sslEnabled: false on the S3 client — traffic to the endpoint is plain HTTP. (Public reads via aws_s3_url may still be HTTPS depending on that URL.)
3. Upload is not transactional with the row write. saveFiles() (S3 put) runs before repo.create(). A failed Mongo write leaves an orphaned object in the bucket (no GC). KYC is the only flow with a compensating delete on the consuming document's failure (§6.1).
4. deleteFile is fire-and-forget and incomplete. BucketService.deleteFile does not await the deleteObject().promise(), derives the key from the last path segment only, and ignores dir sub-folders — so resized variants (433/…, etc.) and any nested objects are never deleted. FileUploadService.delete swallows errors (console.error) and still soft-deletes the row.
5. refId is not a persisted schema prop (§2.1). findByRefId relies on it being spread onto the document at upload time. The reliable link patterns are owner-stores-_id (§6.1/§6.2).
6. Single-vs-multi mismatch. uploadFile ignores the files array and only handles one file; multi-file uploads use the domain receipt mutations.
7. No MIME/type allow-list — only the global size/count guard (maxFiles 25, 100 MB) applies.
8. Training certs bypass the catalogue — they store a raw bucket-key string on TrainingCertificate.certificateUrl, not a FileUpload._id (§6.4).
9. Admin context bug — uploadFile() reads res.data.updateAsset instead of res.data.uploadFile (§7).
10. Filename is UUID-rewritten in storage; the human name lives only in FileUpload.name. Don't rely on the bucket key to recover the original filename.