Skip to main content

Cloud Upload API Documentation

Overview​

The Lilt Cloud Upload API provides secure file upload capabilities using S3-compatible presigned URLs. This system supports both single file uploads and multipart uploads for large files, enabling efficient and secure file transfers directly to cloud storage.

Key Features​

  • Direct S3 Uploads: Files are uploaded directly to S3-compatible storage, reducing server load
  • Presigned URL Security: Time-limited, secure URLs with no exposed credentials
  • Multipart Upload Support: Efficient handling of large files through chunked uploads
  • Flexible Client Support: Use any HTTP client library in your preferred programming language
  • Managed Infrastructure: LILT handles all bucket configuration, CORS setup, and storage management

How It Works​

The upload process uses AWS S3-compatible presigned URLs for secure file transfers:

  1. Request Upload Parameters: Call the Lilt API to get presigned URL and upload parameters
  2. Upload Directly to Storage: Use the presigned URL to upload files directly to S3-compatible storage
  3. Complete Upload:
    • For multipart uploads, notify the API when all parts are uploaded
    • Poll for antivirus scan completion to get the File ID which you can use for other endpoints (add to jobs or projects)

API Endpoints​

1. Initiate Single File Upload​

Endpoint: POST /v2/upload/s3/params or GET /v2/upload/s3/params

Initiates a single file upload and returns presigned URL for direct upload to storage.

Request Body​

{
    "filename": "document.pdf",
    "type": "application/pdf",
    "metadata": {
        "size": 1048576,
        "category": "SOURCE",
        "uuid": "123e4567-e89b-12d3-a456-426614174000"
    }
}

Parameters​

FieldTypeRequiredDescription
filenamestringYesFile name including extension
typestringYesMIME type of the file
metadata.sizeintegerNoFile size in bytes
metadata.categorystringNoFile category (SOURCE or REFERENCE)
metadata.uuidstringNoUnique identifier for the file

Response​

{
    "url": "https://storage.example.com/bucket/path/file.pdf?AWSAccessKeyId=...",
    "method": "PUT",
    "filename": "text2.txt",
    "contentType": "text/plain",
    "metadata": {
        "size": 1048576,
        "category": "SOURCE",
        "uuid": "123e4567-e89b-12d3-a456-426614174000"
    },
    "upload": {
        "createdAt": "2025-07-08T11:11:27.674Z",
        "updatedAt": "2025-07-08T11:11:27.674Z",
        "isDeleted": false,
        "deletedAt": null,
        "id": 362,
        "UserId": 10727,
        "OrganizationId": 682,
        "fileLocation": "gs://.../text2.txt",
        "status": "UPLOADING",
        "totalBytes": 333,
        "uploadedBytes": 0,
        "category": "SOURCE"
    }
}

2. Initiate Multipart Upload​

Endpoint: POST /v2/upload/s3/multipart

Initiates a multipart upload for large files (recommended for files > 100MB).

Make sure your part size is set to 8MB (8388608 bytes).

Request Body​

{
    "filename": "large-file.zip",
    "type": "application/zip",
    "metadata": {
        "size": 104857600,
        "category": "SOURCE"
    }
}

Response​

{
    "uploadId": "abc123def456",
    "key": "uploads/user123/large-file.zip",
    "upload": {
        "createdAt": "2025-07-08T11:11:27.674Z",
        "updatedAt": "2025-07-08T11:11:27.674Z",
        "isDeleted": false,
        "deletedAt": null,
        "id": 362,
        "UserId": 10727,
        "OrganizationId": 682,
        "fileLocation": "gs://.../text2.txt",
        "status": "UPLOADING",
        "totalBytes": 333,
        "uploadedBytes": 0,
        "category": "SOURCE"
    }
}

3. Get Upload Part URL​

Endpoint: GET /v2/upload/s3/multipart/{uploadId}/{partNumber}

Retrieves a presigned URL for uploading a specific part of a multipart upload.

Parameters​

ParameterTypeRequiredDescription
uploadIdstringYesMultipart upload ID from initiate response
partNumberintegerYesPart number (1-based, 1-10000)
s3KeystringYesUpload key from initiate response
sizeintegerYesSize of the file

Example Request​

GET /v2/upload/s3/multipart/abc123def456/1?s3Key=uploads/user123/large-file.zip&size=104857600

Response​

{
    "url": "https://storage.example.com/bucket/path/file.zip?partNumber=1&uploadId=...",
    "method": "PUT"
}

4. Complete Multipart Upload​

Endpoint: POST /v2/upload/s3/multipart/{uploadId}/complete

Completes a multipart upload by providing information about all uploaded parts.

Parameters​

ParameterTypeRequiredDescription
uploadIdstringYesMultipart upload ID
s3KeystringYesUpload key from initiate response

Request Body​

{
    "parts": [
        {
            "ETag": "\"abc123def456\"",
            "PartNumber": 1
        },
        {
            "ETag": "\"def789ghi012\"",
            "PartNumber": 2
        }
    ]
}

Response​

{
    "success": true,
    "location": "https://storage.example.com/bucket/uploads/user123/large-file.zip"
}

5. Cancel Multipart Upload​

Endpoint: DELETE /v2/upload/s3/multipart/{uploadId}

Cancels a multipart upload and cleans up any uploaded parts.

Parameters​

ParameterTypeRequiredDescription
uploadIdstringYesMultipart upload ID to cancel
s3KeystringYesUpload key from initiate response

Response​

{
    "success": true,
    "message": "Multipart upload cancelled successfully"
}

Upload Workflows​

Simple Upload Flow​

1. POST /v2/upload/s3/params → Get presigned URL and upload object
2. PUT to presigned URL → Upload file directly to S3
3. Poll GET /v2/upload/:uploadId for AV scan completion
4. File is ready for use when upload.status=SUCCESS and upload.FileId is a number

Multipart Upload Flow​

1. POST /v2/upload/s3/multipart → Get upload ID and key
2. For each part:
   - GET /v2/upload/s3/multipart/{uploadId}/{partNumber} → Get part URL
   - PUT to part URL → Upload part to S3
   - Save ETag from response
3. POST /v2/upload/s3/multipart/{uploadId}/complete → Complete upload
4. Poll GET /v2/upload/:uploadId for AV scan completion
5. File is ready for use when upload.status=SUCCESS and upload.FileId is a number

Implementation Examples​

Node.js Example​

const axios = require("axios");
const fs = require("fs");

// Initialize axios with base URL
const api = axios.create({
    baseURL: "https://lilt.com/v2",
    headers: {
        Authorization: "Bearer YOUR_API_KEY"
    }
});

// Single file upload
async function uploadFile(filePath, filename, contentType) {
    const fileBuffer = fs.readFileSync(filePath);

    // 1. Initiate upload
    const { data: uploadParams } = await api.post("/upload/s3/params", {
        filename,
        type: contentType,
        metadata: {
            size: fileBuffer.length
        }
    });

    // 2. Upload to S3 using presigned URL
    await axios.put(uploadParams.url, fileBuffer, {
        headers: {
            "Content-Type": contentType,
            ...uploadParams.headers
        }
    });

    console.log("Upload successful");
}

// Multipart upload for large files
async function uploadLargeFile(filePath, filename, contentType) {
    const fileBuffer = fs.readFileSync(filePath);
    const chunkSize = 5 * 1024 * 1024; // 5MB chunks

    // 1. Initiate multipart upload
    const { data: initResponse } = await api.post("/upload/s3/multipart", {
        filename,
        type: contentType,
        metadata: {
            size: fileBuffer.length
        }
    });

    const { uploadId, key } = initResponse;
    const parts = [];

    // 2. Upload each part
    for (let i = 0; i < fileBuffer.length; i += chunkSize) {
        const partNumber = Math.floor(i / chunkSize) + 1;
        const chunk = fileBuffer.slice(i, i + chunkSize);

        // Get presigned URL for this part
        const { data: partParams } = await api.get(
            `/upload/s3/multipart/${uploadId}/${partNumber}`,
            {
                params: {
                    s3Key: key,
                    size: chunk.length
                }
            }
        );

        // Upload part
        const response = await axios.put(partParams.url, chunk, {
            headers: {
                "Content-Type": contentType
            }
        });

        parts.push({
            ETag: response.headers.etag,
            PartNumber: partNumber
        });
    }

    // 3. Complete multipart upload
    await api.post(
        `/upload/s3/multipart/${uploadId}/complete`,
        {
            parts
        },
        {
            params: {
                s3Key: key
            }
        }
    );

    console.log("Multipart upload successful");
}

Python Example​

import requests
import os
from typing import List, Dict

class LiltUploadClient:
    def __init__(self, api_key: str, base_url: str = "https://lilt.com/v2"):
        self.api_key = api_key
        self.base_url = base_url
        self.headers = {
            'Authorization': f'Bearer {api_key}',
            'Content-Type': 'application/json'
        }

    def upload_file(self, file_path: str, filename: str, content_type: str):
        """Upload a single file"""
        with open(file_path, 'rb') as f:
            file_data = f.read()

        # 1. Initiate upload
        response = requests.post(
            f"{self.base_url}/upload/s3/params",
            json={
                "filename": filename,
                "type": content_type,
                "metadata": {
                    "size": len(file_data)
                }
            },
            headers=self.headers
        )
        response.raise_for_status()
        upload_params = response.json()

        # 2. Upload to S3
        upload_headers = {
            'Content-Type': content_type,
            **upload_params.get('headers', {})
        }

        upload_response = requests.put(
            upload_params['url'],
            data=file_data,
            headers=upload_headers
        )
        upload_response.raise_for_status()

        print("Upload successful")

    def upload_large_file(self, file_path: str, filename: str, content_type: str, chunk_size: int = 5 * 1024 * 1024):
        """Upload a large file using multipart upload"""
        file_size = os.path.getsize(file_path)

        # 1. Initiate multipart upload
        response = requests.post(
            f"{self.base_url}/upload/s3/multipart",
            json={
                "filename": filename,
                "type": content_type,
                "metadata": {
                    "size": file_size
                }
            },
            headers=self.headers
        )
        response.raise_for_status()
        init_response = response.json()

        upload_id = init_response['uploadId']
        key = init_response['key']
        parts = []

        # 2. Upload each part
        with open(file_path, 'rb') as f:
            part_number = 1
            while True:
                chunk = f.read(chunk_size)
                if not chunk:
                    break

                # Get presigned URL for this part
                part_response = requests.get(
                    f"{self.base_url}/upload/s3/multipart/{upload_id}/{part_number}",
                    params={
                        's3Key': key,
                        'size': len(chunk)
                    },
                    headers=self.headers
                )
                part_response.raise_for_status()
                part_params = part_response.json()

                # Upload part
                upload_response = requests.put(
                    part_params['url'],
                    data=chunk,
                    headers={'Content-Type': content_type}
                )
                upload_response.raise_for_status()

                parts.append({
                    'ETag': upload_response.headers['etag'],
                    'PartNumber': part_number
                })

                part_number += 1

        # 3. Complete multipart upload
        complete_response = requests.post(
            f"{self.base_url}/upload/s3/multipart/{upload_id}/complete",
            json={'parts': parts},
            params={'s3Key': key},
            headers=self.headers
        )
        complete_response.raise_for_status()

        print("Multipart upload successful")

# Usage
client = LiltUploadClient('your-api-key')
client.upload_file('/path/to/file.pdf', 'document.pdf', 'application/pdf')

Java Example​

import java.io.*;
import java.net.http.*;
import java.util.*;
import com.fasterxml.jackson.databind.ObjectMapper;

public class LiltUploadClient {
    private final String apiKey;
    private final String baseUrl;
    private final HttpClient httpClient;
    private final ObjectMapper objectMapper;

    public LiltUploadClient(String apiKey, String baseUrl) {
        this.apiKey = apiKey;
        this.baseUrl = baseUrl != null ? baseUrl : "https://lilt.com/v2";
        this.httpClient = HttpClient.newHttpClient();
        this.objectMapper = new ObjectMapper();
    }

    public void uploadFile(String filePath, String filename, String contentType) throws Exception {
        byte[] fileData = Files.readAllBytes(Paths.get(filePath));

        // 1. Initiate upload
        Map<String, Object> uploadRequest = new HashMap<>();
        uploadRequest.put("filename", filename);
        uploadRequest.put("type", contentType);

        Map<String, Object> metadata = new HashMap<>();
        metadata.put("size", fileData.length);
        uploadRequest.put("metadata", metadata);

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(baseUrl + "/upload/s3/params"))
            .header("Authorization", "Bearer " + apiKey)
            .header("Content-Type", "application/json")
            .POST(HttpRequest.BodyPublishers.ofString(objectMapper.writeValueAsString(uploadRequest)))
            .build();

        HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());

        if (response.statusCode() != 200) {
            throw new RuntimeException("Failed to initiate upload: " + response.body());
        }

        Map<String, Object> uploadParams = objectMapper.readValue(response.body(), Map.class);

        // 2. Upload to S3
        HttpRequest.Builder uploadRequestBuilder = HttpRequest.newBuilder()
            .uri(URI.create((String) uploadParams.get("url")))
            .header("Content-Type", contentType)
            .PUT(HttpRequest.BodyPublishers.ofByteArray(fileData));

        // Add any additional headers
        Map<String, String> headers = (Map<String, String>) uploadParams.get("headers");
        if (headers != null) {
            headers.forEach(uploadRequestBuilder::header);
        }

        HttpRequest uploadRequest = uploadRequestBuilder.build();
        HttpResponse<String> uploadResponse = httpClient.send(uploadRequest, HttpResponse.BodyHandlers.ofString());

        if (uploadResponse.statusCode() != 200) {
            throw new RuntimeException("Failed to upload file: " + uploadResponse.body());
        }

        System.out.println("Upload successful");
    }

    public void uploadLargeFile(String filePath, String filename, String contentType) throws Exception {
        File file = new File(filePath);
        long fileSize = file.length();
        int chunkSize = 5 * 1024 * 1024; // 5MB chunks

        // 1. Initiate multipart upload
        Map<String, Object> uploadRequest = new HashMap<>();
        uploadRequest.put("filename", filename);
        uploadRequest.put("type", contentType);

        Map<String, Object> metadata = new HashMap<>();
        metadata.put("size", fileSize);
        uploadRequest.put("metadata", metadata);

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(baseUrl + "/upload/s3/multipart"))
            .header("Authorization", "Bearer " + apiKey)
            .header("Content-Type", "application/json")
            .POST(HttpRequest.BodyPublishers.ofString(objectMapper.writeValueAsString(uploadRequest)))
            .build();

        HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
        Map<String, Object> initResponse = objectMapper.readValue(response.body(), Map.class);

        String uploadId = (String) initResponse.get("uploadId");
        String key = (String) initResponse.get("key");
        List<Map<String, Object>> parts = new ArrayList<>();

        // 2. Upload each part
        try (FileInputStream fis = new FileInputStream(file)) {
            byte[] buffer = new byte[chunkSize];
            int partNumber = 1;
            int bytesRead;

            while ((bytesRead = fis.read(buffer)) != -1) {
                byte[] chunk = bytesRead == chunkSize ? buffer : Arrays.copyOf(buffer, bytesRead);

                // Get presigned URL for this part
                String partUrl = String.format("%s/upload/s3/multipart/%s/%d?s3Key=%s&size=%d",
                    baseUrl, uploadId, partNumber, key, chunk.length);

                HttpRequest partRequest = HttpRequest.newBuilder()
                    .uri(URI.create(partUrl))
                    .header("Authorization", "Bearer " + apiKey)
                    .GET()
                    .build();

                HttpResponse<String> partResponse = httpClient.send(partRequest, HttpResponse.BodyHandlers.ofString());
                Map<String, Object> partParams = objectMapper.readValue(partResponse.body(), Map.class);

                // Upload part
                HttpRequest uploadPartRequest = HttpRequest.newBuilder()
                    .uri(URI.create((String) partParams.get("url")))
                    .header("Content-Type", contentType)
                    .PUT(HttpRequest.BodyPublishers.ofByteArray(chunk))
                    .build();

                HttpResponse<String> uploadPartResponse = httpClient.send(uploadPartRequest, HttpResponse.BodyHandlers.ofString());

                Map<String, Object> part = new HashMap<>();
                part.put("ETag", uploadPartResponse.headers().firstValue("etag").orElse(""));
                part.put("PartNumber", partNumber);
                parts.add(part);

                partNumber++;
            }
        }

        // 3. Complete multipart upload
        Map<String, Object> completeRequest = new HashMap<>();
        completeRequest.put("parts", parts);

        HttpRequest completeHttpRequest = HttpRequest.newBuilder()
            .uri(URI.create(String.format("%s/upload/s3/multipart/%s/complete?s3Key=%s", baseUrl, uploadId, key)))
            .header("Authorization", "Bearer " + apiKey)
            .header("Content-Type", "application/json")
            .POST(HttpRequest.BodyPublishers.ofString(objectMapper.writeValueAsString(completeRequest)))
            .build();

        HttpResponse<String> completeResponse = httpClient.send(completeHttpRequest, HttpResponse.BodyHandlers.ofString());

        if (completeResponse.statusCode() != 200) {
            throw new RuntimeException("Failed to complete upload: " + completeResponse.body());
        }

        System.out.println("Multipart upload successful");
    }
}

// Usage
LiltUploadClient client = new LiltUploadClient("your-api-key", null);
client.uploadFile("/path/to/file.pdf", "document.pdf", "application/pdf");

Best Practices​

File Size Recommendations​

  • Small files (< 100MB): Use single file upload (POST /upload/s3/params)
  • Large files (> 100MB): Use multipart upload (POST /upload/s3/multipart)
  • Chunk size: Use 8MB chunks for multipart uploads

Error Handling​

Implement retry logic for network errors:

async function uploadWithRetry(uploadFn, maxRetries = 3) {
    for (let i = 0; i < maxRetries; i++) {
        try {
            await uploadFn();
            return;
        } catch (error) {
            if (i === maxRetries - 1) throw error;
            await new Promise(resolve =>
                setTimeout(resolve, 1000 * Math.pow(2, i))
            );
        }
    }
}

Security Considerations​

  • Presigned URLs expire after a configurable time (typically 1 hour)
  • URLs are single-use for uploads
  • All uploads are validated against the original request parameters
  • HTTPS is required for all upload operations

Common Error Responses​

400 Bad Request​

{
    "error": {
        "message": "Invalid file type",
        "code": "INVALID_FILE_TYPE"
    }
}

413 Payload Too Large​

{
    "error": {
        "message": "File size exceeds maximum allowed size",
        "code": "FILE_TOO_LARGE",
        "details": {
            "maxSize": 104857600,
            "actualSize": 209715200
        }
    }
}

403 Forbidden​

{
    "error": {
        "message": "Upload URL has expired",
        "code": "URL_EXPIRED"
    }
}

Support​

For questions or issues with the Cloud Upload API, please contact support@lilt.com or refer to the main API documentation.