veza/FILE_UPLOAD_FORMAT.md

# File Upload Format Standardization

## INT-015: Add file upload format standardization

**Date**: 2025-12-25  
**Status**: Completed

## Overview

This document defines the standardized format for all file uploads in the Veza platform. It ensures consistency between backend and frontend, making upload handling predictable and maintainable.

## Standard Upload Request Format

All file uploads use `multipart/form-data` with the following standardized field names:

### Required Fields

- **`file`** (file, required): The actual file being uploaded

### Optional Metadata Fields

- **`title`** (string, optional): Title of the content
- **`artist`** (string, optional): Artist name (for audio files)
- **`album`** (string, optional): Album name (for audio files)
- **`genre`** (string, optional): Genre
- **`year`** (integer, optional): Year
- **`description`** (string, optional): Description

### File Type and Context Fields

- **`file_type`** (string, optional): Type of file - `"audio"`, `"image"`, or `"video"` (auto-detected if not provided)
- **`track_id`** (UUID, optional): Track ID if updating an existing track
- **`is_public`** (boolean, optional): Public visibility (default: `false`)
- **`metadata`** (string, optional): JSON string with additional metadata

### Example Request

```javascript
const formData = new FormData();
formData.append('file', file);
formData.append('title', 'My Track');
formData.append('artist', 'Artist Name');
formData.append('album', 'Album Name');
formData.append('genre', 'Electronic');
formData.append('year', '2025');
formData.append('is_public', 'true');

await apiClient.post('/tracks', formData, {
  headers: {
    'Content-Type': 'multipart/form-data',
  },
});
```

## Standard Upload Response Format

All upload responses follow this standardized structure:

```json
{
  "success": true,
  "data": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "track_id": "660e8400-e29b-41d4-a716-446655440001",
    "file_name": "track.mp3",
    "file_size": 5242880,
    "file_type": "audio",
    "mime_type": "audio/mpeg",
    "checksum": "sha256:abc123...",
    "status": "completed",
    "progress": 100,
    "bytes_uploaded": 5242880,
    "url": "https://cdn.example.com/tracks/550e8400...",
    "thumbnail_url": "https://cdn.example.com/thumbnails/550e8400...",
    "storage_path": "tracks/2025/12/550e8400...",
    "storage_provider": "s3",
    "is_processed": true,
    "processed_at": "2025-12-25T10:30:00Z",
    "processing_error": null,
    "virus_scanned": true,
    "virus_scan_result": "clean",
    "virus_scanned_at": "2025-12-25T10:29:45Z",
    "created_at": "2025-12-25T10:29:30Z",
    "updated_at": "2025-12-25T10:30:00Z",
    "expires_at": null
  }
}
```

### Response Fields

- **`id`** (UUID): Upload ID
- **`track_id`** (UUID, optional): Track ID (if applicable)
- **`file_name`** (string): Original filename
- **`file_size`** (int64): File size in bytes
- **`file_type`** (string): File type (`"audio"`, `"image"`, `"video"`)
- **`mime_type`** (string): MIME type
- **`checksum`** (string): File checksum (SHA-256)
- **`status`** (string): Upload status (`"pending"`, `"uploading"`, `"processing"`, `"completed"`, `"failed"`, `"cancelled"`)
- **`progress`** (int): Progress percentage (0-100)
- **`bytes_uploaded`** (int64): Bytes uploaded so far
- **`url`** (string): Public URL (if available)
- **`thumbnail_url`** (string, optional): Thumbnail URL (if applicable)
- **`storage_path`** (string): Storage path
- **`storage_provider`** (string): Storage provider (`"s3"`, `"local"`, etc.)
- **`is_processed`** (bool): Whether file has been processed
- **`processed_at`** (string, optional): Processing completion time (ISO 8601)
- **`processing_error`** (string, optional): Processing error (if any)
- **`virus_scanned`** (bool): Whether file was scanned
- **`virus_scan_result`** (string, optional): Scan result (`"clean"`, `"infected"`, `"error"`)
- **`virus_scanned_at`** (string, optional): Scan timestamp (ISO 8601)
- **`created_at`** (string): Upload creation time (ISO 8601)
- **`updated_at`** (string): Last update time (ISO 8601)
- **`expires_at`** (string, optional): Expiration time (ISO 8601, if applicable)

## Upload Status Values

| Status | Description |
|--------|-------------|
| `pending` | Upload queued, not started |
| `uploading` | File is being uploaded |
| `processing` | File is being processed (transcoding, etc.) |
| `completed` | Upload and processing completed successfully |
| `failed` | Upload or processing failed |
| `cancelled` | Upload was cancelled |

## Error Response Format

Upload errors follow the standard error response format:

```json
{
  "success": false,
  "error": {
    "code": "FILE_TOO_LARGE",
    "message": "File size exceeds maximum allowed size of 100MB",
    "details": {
      "file_size": 104857600,
      "max_size": 100000000,
      "field": "file"
    }
  }
}
```

### Error Codes

| Code | HTTP Status | Description |
|------|-------------|-------------|
| `FILE_REQUIRED` | 400 | No file provided |
| `FILE_TOO_LARGE` | 413 | File size exceeds maximum |
| `INVALID_FILE_TYPE` | 415 | File type not supported |
| `INVALID_FILE_FORMAT` | 400 | File format is invalid |
| `VIRUS_DETECTED` | 422 | Virus detected in file |
| `VIRUS_SCAN_FAILED` | 503 | Virus scan failed |
| `VIRUS_SCAN_UNAVAILABLE` | 503 | Virus scanning service unavailable |
| `QUOTA_EXCEEDED` | 403 | Upload quota exceeded |
| `UPLOAD_FAILED` | 500 | Upload failed |
| `PROCESSING_FAILED` | 500 | Processing failed |
| `TOO_MANY_CONCURRENT_UPLOADS` | 503 | Too many concurrent uploads |
| `INVALID_METADATA` | 400 | Invalid metadata format |

## File Type Limits

### Audio Files

- **Max Size**: 100MB (104,857,600 bytes)
- **Allowed MIME Types**: 
  - `audio/mpeg`
  - `audio/mp3`
  - `audio/wav`
  - `audio/flac`
  - `audio/aac`
  - `audio/ogg`
  - `audio/m4a`
- **Allowed Extensions**: `.mp3`, `.wav`, `.flac`, `.aac`, `.ogg`, `.m4a`

### Image Files

- **Max Size**: 10MB (10,485,760 bytes)
- **Allowed MIME Types**:
  - `image/jpeg`
  - `image/png`
  - `image/gif`
  - `image/webp`
  - `image/svg+xml`
- **Allowed Extensions**: `.jpg`, `.jpeg`, `.png`, `.gif`, `.webp`, `.svg`

### Video Files

- **Max Size**: 500MB (524,288,000 bytes)
- **Allowed MIME Types**:
  - `video/mp4`
  - `video/webm`
  - `video/ogg`
  - `video/avi`
- **Allowed Extensions**: `.mp4`, `.webm`, `.ogg`, `.avi`

## Upload Progress

For long-running uploads, progress can be tracked:

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "uploading",
  "progress": 45,
  "bytes_uploaded": 2359296,
  "total_bytes": 5242880,
  "estimated_time_remaining": 30,
  "message": "Uploading...",
  "updated_at": "2025-12-25T10:30:00Z"
}
```

## Batch Upload

For multiple file uploads:

```json
{
  "total_files": 5,
  "successful": 4,
  "failed": 1,
  "results": [
    {
      "index": 1,
      "file_name": "track1.mp3",
      "file_size": 5242880,
      "file_type": "audio",
      "status": "completed",
      "upload_id": "550e8400-e29b-41d4-a716-446655440000"
    },
    {
      "index": 2,
      "file_name": "track2.mp3",
      "file_size": 10485760,
      "file_type": "audio",
      "status": "failed",
      "error": "File too large"
    }
  ],
  "errors": []
}
```

## Backend Implementation

### Using Standard Types

```go
import "veza-backend-api/internal/upload"

// Parse upload request
var req upload.StandardUploadRequest
if err := c.ShouldBind(&req); err != nil {
    RespondWithAppError(c, apperrors.New(apperrors.ErrCodeValidation, err.Error()))
    return
}

// Create response
response := &upload.StandardUploadResponse{
    ID:        uploadID,
    FileName:  fileHeader.Filename,
    FileSize:  fileSize,
    FileType:  "audio",
    Status:    upload.UploadStatusCompleted,
    CreatedAt: time.Now(),
}

RespondSuccess(c, http.StatusCreated, response)
```

### Error Handling

```go
// Return standardized error
RespondWithAppError(c, apperrors.New(
    apperrors.ErrCodeValidation,
    "File size exceeds maximum allowed size",
    map[string]interface{}{
        "code": upload.ErrorCodeFileTooLarge,
        "file_size": fileSize,
        "max_size": maxSize,
    },
))
```

## Frontend Implementation

### Upload Request

```typescript
import { apiClient } from '@/services/api/client';

interface UploadMetadata {
  title?: string;
  artist?: string;
  album?: string;
  genre?: string;
  year?: number;
  description?: string;
  is_public?: boolean;
}

async function uploadFile(
  file: File,
  metadata: UploadMetadata = {},
  onProgress?: (progress: number) => void,
): Promise<StandardUploadResponse> {
  const formData = new FormData();
  formData.append('file', file);
  
  if (metadata.title) formData.append('title', metadata.title);
  if (metadata.artist) formData.append('artist', metadata.artist);
  if (metadata.album) formData.append('album', metadata.album);
  if (metadata.genre) formData.append('genre', metadata.genre);
  if (metadata.year) formData.append('year', metadata.year.toString());
  if (metadata.description) formData.append('description', metadata.description);
  if (metadata.is_public !== undefined) {
    formData.append('is_public', metadata.is_public.toString());
  }

  const response = await apiClient.post<StandardUploadResponse>(
    '/tracks',
    formData,
    {
      headers: {
        'Content-Type': 'multipart/form-data',
      },
      onUploadProgress: (progressEvent) => {
        if (progressEvent.total && onProgress) {
          const progress = Math.round(
            (progressEvent.loaded * 100) / progressEvent.total,
          );
          onProgress(progress);
        }
      },
    },
  );

  return response.data;
}
```

### Type Definitions

```typescript
interface StandardUploadResponse {
  id: string;
  track_id?: string;
  file_name: string;
  file_size: number;
  file_type: 'audio' | 'image' | 'video';
  mime_type: string;
  checksum: string;
  status: 'pending' | 'uploading' | 'processing' | 'completed' | 'failed' | 'cancelled';
  progress: number;
  bytes_uploaded: number;
  url: string;
  thumbnail_url?: string;
  storage_path: string;
  storage_provider: string;
  is_processed: boolean;
  processed_at?: string;
  processing_error?: string;
  virus_scanned: boolean;
  virus_scan_result?: 'clean' | 'infected' | 'error';
  virus_scanned_at?: string;
  created_at: string;
  updated_at: string;
  expires_at?: string;
}
```

## Best Practices

### For Backend Developers

1. **Always Use Standard Types**
   ```go
   var req upload.StandardUploadRequest
   ```

2. **Validate File Size and Type**
   ```go
   if fileSize > maxSize {
       return upload.ErrorCodeFileTooLarge
   }
   ```

3. **Use Standard Error Codes**
   ```go
   errorCode := upload.ErrorCodeFileTooLarge
   ```

4. **Return Standard Response Format**
   ```go
   response := &upload.StandardUploadResponse{...}
   RespondSuccess(c, http.StatusCreated, response)
   ```

### For Frontend Developers

1. **Use Standard Field Names**
   ```typescript
   formData.append('file', file);
   formData.append('title', title);
   ```

2. **Handle Progress Updates**
   ```typescript
   onUploadProgress: (progressEvent) => {
     const progress = Math.round(
       (progressEvent.loaded * 100) / progressEvent.total,
     );
     onProgress(progress);
   }
   ```

3. **Handle Errors Properly**
   ```typescript
   if (error.response?.data?.error?.code === 'FILE_TOO_LARGE') {
     showError('File is too large');
   }
   ```

4. **Validate Before Upload**
   ```typescript
   if (file.size > maxSize) {
     throw new Error('File too large');
   }
   ```

## Migration Guide

### Legacy Format (Deprecated)

```go
// Old format
type UploadRequest struct {
    TrackID  uuid.UUID `form:"track_id"`
    FileType string    `form:"file_type"`
    Title    string    `form:"title"`
}
```

### Standardized Format

```go
// New format
import "veza-backend-api/internal/upload"

var req upload.StandardUploadRequest
```

## References

- `DATETIME_STANDARD.md` - Date/time format specification
- `ERROR_RESPONSE_STANDARD.md` - Error format specification
- `REQUEST_RESPONSE_VALIDATION_GUIDE.md` - Validation guide
- `veza-backend-api/internal/upload/types.go` - Backend types
- `apps/web/src/types/upload.ts` - Frontend types (to be created)

---

**Last Updated**: 2025-12-25  
**Maintained By**: Veza Backend Team