469 lines
12 KiB
Markdown
469 lines
12 KiB
Markdown
# File Upload Format Standardization
|
|
|
|
## INT-015: Add file upload format standardization
|
|
|
|
**Date**: 2025-12-25
|
|
**Status**: Completed
|
|
|
|
## Overview
|
|
|
|
This document defines the standardized format for all file uploads in the Veza platform. It ensures consistency between backend and frontend, making upload handling predictable and maintainable.
|
|
|
|
## Standard Upload Request Format
|
|
|
|
All file uploads use `multipart/form-data` with the following standardized field names:
|
|
|
|
### Required Fields
|
|
|
|
- **`file`** (file, required): The actual file being uploaded
|
|
|
|
### Optional Metadata Fields
|
|
|
|
- **`title`** (string, optional): Title of the content
|
|
- **`artist`** (string, optional): Artist name (for audio files)
|
|
- **`album`** (string, optional): Album name (for audio files)
|
|
- **`genre`** (string, optional): Genre
|
|
- **`year`** (integer, optional): Year
|
|
- **`description`** (string, optional): Description
|
|
|
|
### File Type and Context Fields
|
|
|
|
- **`file_type`** (string, optional): Type of file - `"audio"`, `"image"`, or `"video"` (auto-detected if not provided)
|
|
- **`track_id`** (UUID, optional): Track ID if updating an existing track
|
|
- **`is_public`** (boolean, optional): Public visibility (default: `false`)
|
|
- **`metadata`** (string, optional): JSON string with additional metadata
|
|
|
|
### Example Request
|
|
|
|
```javascript
|
|
const formData = new FormData();
|
|
formData.append('file', file);
|
|
formData.append('title', 'My Track');
|
|
formData.append('artist', 'Artist Name');
|
|
formData.append('album', 'Album Name');
|
|
formData.append('genre', 'Electronic');
|
|
formData.append('year', '2025');
|
|
formData.append('is_public', 'true');
|
|
|
|
await apiClient.post('/tracks', formData, {
|
|
headers: {
|
|
'Content-Type': 'multipart/form-data',
|
|
},
|
|
});
|
|
```
|
|
|
|
## Standard Upload Response Format
|
|
|
|
All upload responses follow this standardized structure:
|
|
|
|
```json
|
|
{
|
|
"success": true,
|
|
"data": {
|
|
"id": "550e8400-e29b-41d4-a716-446655440000",
|
|
"track_id": "660e8400-e29b-41d4-a716-446655440001",
|
|
"file_name": "track.mp3",
|
|
"file_size": 5242880,
|
|
"file_type": "audio",
|
|
"mime_type": "audio/mpeg",
|
|
"checksum": "sha256:abc123...",
|
|
"status": "completed",
|
|
"progress": 100,
|
|
"bytes_uploaded": 5242880,
|
|
"url": "https://cdn.example.com/tracks/550e8400...",
|
|
"thumbnail_url": "https://cdn.example.com/thumbnails/550e8400...",
|
|
"storage_path": "tracks/2025/12/550e8400...",
|
|
"storage_provider": "s3",
|
|
"is_processed": true,
|
|
"processed_at": "2025-12-25T10:30:00Z",
|
|
"processing_error": null,
|
|
"virus_scanned": true,
|
|
"virus_scan_result": "clean",
|
|
"virus_scanned_at": "2025-12-25T10:29:45Z",
|
|
"created_at": "2025-12-25T10:29:30Z",
|
|
"updated_at": "2025-12-25T10:30:00Z",
|
|
"expires_at": null
|
|
}
|
|
}
|
|
```
|
|
|
|
### Response Fields
|
|
|
|
- **`id`** (UUID): Upload ID
|
|
- **`track_id`** (UUID, optional): Track ID (if applicable)
|
|
- **`file_name`** (string): Original filename
|
|
- **`file_size`** (int64): File size in bytes
|
|
- **`file_type`** (string): File type (`"audio"`, `"image"`, `"video"`)
|
|
- **`mime_type`** (string): MIME type
|
|
- **`checksum`** (string): File checksum (SHA-256)
|
|
- **`status`** (string): Upload status (`"pending"`, `"uploading"`, `"processing"`, `"completed"`, `"failed"`, `"cancelled"`)
|
|
- **`progress`** (int): Progress percentage (0-100)
|
|
- **`bytes_uploaded`** (int64): Bytes uploaded so far
|
|
- **`url`** (string): Public URL (if available)
|
|
- **`thumbnail_url`** (string, optional): Thumbnail URL (if applicable)
|
|
- **`storage_path`** (string): Storage path
|
|
- **`storage_provider`** (string): Storage provider (`"s3"`, `"local"`, etc.)
|
|
- **`is_processed`** (bool): Whether file has been processed
|
|
- **`processed_at`** (string, optional): Processing completion time (ISO 8601)
|
|
- **`processing_error`** (string, optional): Processing error (if any)
|
|
- **`virus_scanned`** (bool): Whether file was scanned
|
|
- **`virus_scan_result`** (string, optional): Scan result (`"clean"`, `"infected"`, `"error"`)
|
|
- **`virus_scanned_at`** (string, optional): Scan timestamp (ISO 8601)
|
|
- **`created_at`** (string): Upload creation time (ISO 8601)
|
|
- **`updated_at`** (string): Last update time (ISO 8601)
|
|
- **`expires_at`** (string, optional): Expiration time (ISO 8601, if applicable)
|
|
|
|
## Upload Status Values
|
|
|
|
| Status | Description |
|
|
|--------|-------------|
|
|
| `pending` | Upload queued, not started |
|
|
| `uploading` | File is being uploaded |
|
|
| `processing` | File is being processed (transcoding, etc.) |
|
|
| `completed` | Upload and processing completed successfully |
|
|
| `failed` | Upload or processing failed |
|
|
| `cancelled` | Upload was cancelled |
|
|
|
|
## Error Response Format
|
|
|
|
Upload errors follow the standard error response format:
|
|
|
|
```json
|
|
{
|
|
"success": false,
|
|
"error": {
|
|
"code": "FILE_TOO_LARGE",
|
|
"message": "File size exceeds maximum allowed size of 100MB",
|
|
"details": {
|
|
"file_size": 104857600,
|
|
"max_size": 100000000,
|
|
"field": "file"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Error Codes
|
|
|
|
| Code | HTTP Status | Description |
|
|
|------|-------------|-------------|
|
|
| `FILE_REQUIRED` | 400 | No file provided |
|
|
| `FILE_TOO_LARGE` | 413 | File size exceeds maximum |
|
|
| `INVALID_FILE_TYPE` | 415 | File type not supported |
|
|
| `INVALID_FILE_FORMAT` | 400 | File format is invalid |
|
|
| `VIRUS_DETECTED` | 422 | Virus detected in file |
|
|
| `VIRUS_SCAN_FAILED` | 503 | Virus scan failed |
|
|
| `VIRUS_SCAN_UNAVAILABLE` | 503 | Virus scanning service unavailable |
|
|
| `QUOTA_EXCEEDED` | 403 | Upload quota exceeded |
|
|
| `UPLOAD_FAILED` | 500 | Upload failed |
|
|
| `PROCESSING_FAILED` | 500 | Processing failed |
|
|
| `TOO_MANY_CONCURRENT_UPLOADS` | 503 | Too many concurrent uploads |
|
|
| `INVALID_METADATA` | 400 | Invalid metadata format |
|
|
|
|
## File Type Limits
|
|
|
|
### Audio Files
|
|
|
|
- **Max Size**: 100MB (104,857,600 bytes)
|
|
- **Allowed MIME Types**:
|
|
- `audio/mpeg`
|
|
- `audio/mp3`
|
|
- `audio/wav`
|
|
- `audio/flac`
|
|
- `audio/aac`
|
|
- `audio/ogg`
|
|
- `audio/m4a`
|
|
- **Allowed Extensions**: `.mp3`, `.wav`, `.flac`, `.aac`, `.ogg`, `.m4a`
|
|
|
|
### Image Files
|
|
|
|
- **Max Size**: 10MB (10,485,760 bytes)
|
|
- **Allowed MIME Types**:
|
|
- `image/jpeg`
|
|
- `image/png`
|
|
- `image/gif`
|
|
- `image/webp`
|
|
- `image/svg+xml`
|
|
- **Allowed Extensions**: `.jpg`, `.jpeg`, `.png`, `.gif`, `.webp`, `.svg`
|
|
|
|
### Video Files
|
|
|
|
- **Max Size**: 500MB (524,288,000 bytes)
|
|
- **Allowed MIME Types**:
|
|
- `video/mp4`
|
|
- `video/webm`
|
|
- `video/ogg`
|
|
- `video/avi`
|
|
- **Allowed Extensions**: `.mp4`, `.webm`, `.ogg`, `.avi`
|
|
|
|
## Upload Progress
|
|
|
|
For long-running uploads, progress can be tracked:
|
|
|
|
```json
|
|
{
|
|
"id": "550e8400-e29b-41d4-a716-446655440000",
|
|
"status": "uploading",
|
|
"progress": 45,
|
|
"bytes_uploaded": 2359296,
|
|
"total_bytes": 5242880,
|
|
"estimated_time_remaining": 30,
|
|
"message": "Uploading...",
|
|
"updated_at": "2025-12-25T10:30:00Z"
|
|
}
|
|
```
|
|
|
|
## Batch Upload
|
|
|
|
For multiple file uploads:
|
|
|
|
```json
|
|
{
|
|
"total_files": 5,
|
|
"successful": 4,
|
|
"failed": 1,
|
|
"results": [
|
|
{
|
|
"index": 1,
|
|
"file_name": "track1.mp3",
|
|
"file_size": 5242880,
|
|
"file_type": "audio",
|
|
"status": "completed",
|
|
"upload_id": "550e8400-e29b-41d4-a716-446655440000"
|
|
},
|
|
{
|
|
"index": 2,
|
|
"file_name": "track2.mp3",
|
|
"file_size": 10485760,
|
|
"file_type": "audio",
|
|
"status": "failed",
|
|
"error": "File too large"
|
|
}
|
|
],
|
|
"errors": []
|
|
}
|
|
```
|
|
|
|
## Backend Implementation
|
|
|
|
### Using Standard Types
|
|
|
|
```go
|
|
import "veza-backend-api/internal/upload"
|
|
|
|
// Parse upload request
|
|
var req upload.StandardUploadRequest
|
|
if err := c.ShouldBind(&req); err != nil {
|
|
RespondWithAppError(c, apperrors.New(apperrors.ErrCodeValidation, err.Error()))
|
|
return
|
|
}
|
|
|
|
// Create response
|
|
response := &upload.StandardUploadResponse{
|
|
ID: uploadID,
|
|
FileName: fileHeader.Filename,
|
|
FileSize: fileSize,
|
|
FileType: "audio",
|
|
Status: upload.UploadStatusCompleted,
|
|
CreatedAt: time.Now(),
|
|
}
|
|
|
|
RespondSuccess(c, http.StatusCreated, response)
|
|
```
|
|
|
|
### Error Handling
|
|
|
|
```go
|
|
// Return standardized error
|
|
RespondWithAppError(c, apperrors.New(
|
|
apperrors.ErrCodeValidation,
|
|
"File size exceeds maximum allowed size",
|
|
map[string]interface{}{
|
|
"code": upload.ErrorCodeFileTooLarge,
|
|
"file_size": fileSize,
|
|
"max_size": maxSize,
|
|
},
|
|
))
|
|
```
|
|
|
|
## Frontend Implementation
|
|
|
|
### Upload Request
|
|
|
|
```typescript
|
|
import { apiClient } from '@/services/api/client';
|
|
|
|
interface UploadMetadata {
|
|
title?: string;
|
|
artist?: string;
|
|
album?: string;
|
|
genre?: string;
|
|
year?: number;
|
|
description?: string;
|
|
is_public?: boolean;
|
|
}
|
|
|
|
async function uploadFile(
|
|
file: File,
|
|
metadata: UploadMetadata = {},
|
|
onProgress?: (progress: number) => void,
|
|
): Promise<StandardUploadResponse> {
|
|
const formData = new FormData();
|
|
formData.append('file', file);
|
|
|
|
if (metadata.title) formData.append('title', metadata.title);
|
|
if (metadata.artist) formData.append('artist', metadata.artist);
|
|
if (metadata.album) formData.append('album', metadata.album);
|
|
if (metadata.genre) formData.append('genre', metadata.genre);
|
|
if (metadata.year) formData.append('year', metadata.year.toString());
|
|
if (metadata.description) formData.append('description', metadata.description);
|
|
if (metadata.is_public !== undefined) {
|
|
formData.append('is_public', metadata.is_public.toString());
|
|
}
|
|
|
|
const response = await apiClient.post<StandardUploadResponse>(
|
|
'/tracks',
|
|
formData,
|
|
{
|
|
headers: {
|
|
'Content-Type': 'multipart/form-data',
|
|
},
|
|
onUploadProgress: (progressEvent) => {
|
|
if (progressEvent.total && onProgress) {
|
|
const progress = Math.round(
|
|
(progressEvent.loaded * 100) / progressEvent.total,
|
|
);
|
|
onProgress(progress);
|
|
}
|
|
},
|
|
},
|
|
);
|
|
|
|
return response.data;
|
|
}
|
|
```
|
|
|
|
### Type Definitions
|
|
|
|
```typescript
|
|
interface StandardUploadResponse {
|
|
id: string;
|
|
track_id?: string;
|
|
file_name: string;
|
|
file_size: number;
|
|
file_type: 'audio' | 'image' | 'video';
|
|
mime_type: string;
|
|
checksum: string;
|
|
status: 'pending' | 'uploading' | 'processing' | 'completed' | 'failed' | 'cancelled';
|
|
progress: number;
|
|
bytes_uploaded: number;
|
|
url: string;
|
|
thumbnail_url?: string;
|
|
storage_path: string;
|
|
storage_provider: string;
|
|
is_processed: boolean;
|
|
processed_at?: string;
|
|
processing_error?: string;
|
|
virus_scanned: boolean;
|
|
virus_scan_result?: 'clean' | 'infected' | 'error';
|
|
virus_scanned_at?: string;
|
|
created_at: string;
|
|
updated_at: string;
|
|
expires_at?: string;
|
|
}
|
|
```
|
|
|
|
## Best Practices
|
|
|
|
### For Backend Developers
|
|
|
|
1. **Always Use Standard Types**
|
|
```go
|
|
var req upload.StandardUploadRequest
|
|
```
|
|
|
|
2. **Validate File Size and Type**
|
|
```go
|
|
if fileSize > maxSize {
|
|
return upload.ErrorCodeFileTooLarge
|
|
}
|
|
```
|
|
|
|
3. **Use Standard Error Codes**
|
|
```go
|
|
errorCode := upload.ErrorCodeFileTooLarge
|
|
```
|
|
|
|
4. **Return Standard Response Format**
|
|
```go
|
|
response := &upload.StandardUploadResponse{...}
|
|
RespondSuccess(c, http.StatusCreated, response)
|
|
```
|
|
|
|
### For Frontend Developers
|
|
|
|
1. **Use Standard Field Names**
|
|
```typescript
|
|
formData.append('file', file);
|
|
formData.append('title', title);
|
|
```
|
|
|
|
2. **Handle Progress Updates**
|
|
```typescript
|
|
onUploadProgress: (progressEvent) => {
|
|
const progress = Math.round(
|
|
(progressEvent.loaded * 100) / progressEvent.total,
|
|
);
|
|
onProgress(progress);
|
|
}
|
|
```
|
|
|
|
3. **Handle Errors Properly**
|
|
```typescript
|
|
if (error.response?.data?.error?.code === 'FILE_TOO_LARGE') {
|
|
showError('File is too large');
|
|
}
|
|
```
|
|
|
|
4. **Validate Before Upload**
|
|
```typescript
|
|
if (file.size > maxSize) {
|
|
throw new Error('File too large');
|
|
}
|
|
```
|
|
|
|
## Migration Guide
|
|
|
|
### Legacy Format (Deprecated)
|
|
|
|
```go
|
|
// Old format
|
|
type UploadRequest struct {
|
|
TrackID uuid.UUID `form:"track_id"`
|
|
FileType string `form:"file_type"`
|
|
Title string `form:"title"`
|
|
}
|
|
```
|
|
|
|
### Standardized Format
|
|
|
|
```go
|
|
// New format
|
|
import "veza-backend-api/internal/upload"
|
|
|
|
var req upload.StandardUploadRequest
|
|
```
|
|
|
|
## References
|
|
|
|
- `DATETIME_STANDARD.md` - Date/time format specification
|
|
- `ERROR_RESPONSE_STANDARD.md` - Error format specification
|
|
- `REQUEST_RESPONSE_VALIDATION_GUIDE.md` - Validation guide
|
|
- `veza-backend-api/internal/upload/types.go` - Backend types
|
|
- `apps/web/src/types/upload.ts` - Frontend types (to be created)
|
|
|
|
---
|
|
|
|
**Last Updated**: 2025-12-25
|
|
**Maintained By**: Veza Backend Team
|
|
|