PHP大文件分块上传与断点续传实战指南

1. 大文件上传的核心挑战

当我们需要在网页中处理500MB以上的大文件上传时，传统的PHP文件上传机制会遇到几个关键瓶颈：

• 内存限制：PHP默认配置中，memory_limit通常设置为128M，无法直接处理大文件
• 执行超时：max_execution_time默认30秒，大文件上传需要更长时间
• 临时存储：upload_tmp_dir的磁盘空间可能不足
• POST限制：post_max_size和upload_max_filesize需要相应调整

2. 基础配置优化方案

2.1 PHP.ini关键参数调整

ini复制; 允许POST的最大数据量，必须大于upload_max_filesize
post_max_size = 600M

; 允许上传的单个文件最大尺寸
upload_max_filesize = 500M

; 脚本执行最长时间（秒）
max_execution_time = 3600

; 脚本解析输入数据允许的最长时间
max_input_time = 3600

; 内存限制
memory_limit = 256M

注意：修改后需要重启Web服务（Apache/Nginx等）才能生效

2.2 临时目录设置

确保临时目录有足够空间并正确配置权限：

bash复制# 查看当前临时目录
php -r "echo ini_get('upload_tmp_dir');"

# 建议设置专用临时目录
mkdir -p /var/www/tmp_uploads
chown www-data:www-data /var/www/tmp_uploads
chmod 750 /var/www/tmp_uploads

3. 分块上传方案

3.1 前端实现分块

使用JavaScript将大文件分割为多个小块（如5MB/块）：

javascript复制// 使用File API读取文件
const chunkSize = 5 * 1024 * 1024; // 5MB
let offset = 0;

function uploadChunk(file, chunkIndex) {
  const chunk = file.slice(offset, offset + chunkSize);
  const formData = new FormData();
  formData.append('chunk', chunk);
  formData.append('chunkIndex', chunkIndex);
  formData.append('totalChunks', Math.ceil(file.size / chunkSize));
  formData.append('originalName', file.name);

  return fetch('/upload.php', {
    method: 'POST',
    body: formData
  }).then(response => response.json());
}

3.2 后端分块处理

PHP接收并合并分块文件：

php复制$targetDir = "uploads/";
$chunkIndex = $_POST['chunkIndex'];
$totalChunks = $_POST['totalChunks'];
$originalName = $_POST['originalName'];

// 为每个上传创建临时目录
$tempDir = $targetDir . 'temp_' . md5($originalName);
if (!file_exists($tempDir)) {
    mkdir($tempDir, 0755, true);
}

// 移动分块到临时目录
$chunkPath = $tempDir . '/' . $originalName . '.part' . $chunkIndex;
move_uploaded_file($_FILES['chunk']['tmp_name'], $chunkPath);

// 检查是否所有分块都已上传
$uploadedChunks = glob($tempDir . "/*.part*");
if (count($uploadedChunks) == $totalChunks) {
    // 合并文件
    $finalPath = $targetDir . $originalName;
    $out = fopen($finalPath, "wb");
    
    for ($i = 0; $i < $totalChunks; $i++) {
        $chunkPath = $tempDir . '/' . $originalName . '.part' . $i;
        $in = fopen($chunkPath, "rb");
        stream_copy_to_stream($in, $out);
        fclose($in);
        unlink($chunkPath);
    }
    
    fclose($out);
    rmdir($tempDir);
    
    echo json_encode(['status' => 'complete']);
} else {
    echo json_encode(['status' => 'chunk_uploaded']);
}

4. 进度监控实现

4.1 使用Session Upload Progress

在php.ini中启用：

ini复制session.upload_progress.enabled = On
session.upload_progress.cleanup = On
session.upload_progress.prefix = "upload_progress_"
session.upload_progress.name = "PHP_SESSION_UPLOAD_PROGRESS"

前端添加隐藏字段：

html复制<input type="hidden" name="<?php echo ini_get('session.upload_progress.name'); ?>" value="file123" />

后端查询进度：

php复制session_start();
$key = ini_get('session.upload_progress.prefix') . $_POST[ini_get('session.upload_progress.name')];
$progress = $_SESSION[$key];
echo json_encode($progress);

4.2 替代方案：自定义进度记录

对于不支持Session Upload Progress的环境：

php复制// 在上传处理脚本中
$progressFile = '/tmp/upload_' . session_id() . '.progress';

function updateProgress($uploaded, $total) {
    file_put_contents($GLOBALS['progressFile'], json_encode([
        'uploaded' => $uploaded,
        'total' => $total,
        'percent' => ($uploaded / $total) * 100
    ]));
}

// 在分块上传成功后调用
updateProgress($chunkIndex * $chunkSize, $totalSize);

5. 断点续传实现

5.1 服务端记录上传状态

php复制function getUploadStatus($fileHash) {
    $statusFile = "/tmp/upload_status_{$fileHash}.json";
    if (file_exists($statusFile)) {
        return json_decode(file_get_contents($statusFile), true);
    }
    return ['chunks' => []];
}

function saveUploadStatus($fileHash, $chunkIndex) {
    $status = getUploadStatus($fileHash);
    $status['chunks'][$chunkIndex] = true;
    file_put_contents("/tmp/upload_status_{$fileHash}.json", json_encode($status));
}

// 在上传前检查
$status = getUploadStatus(md5($originalName));
if (isset($status['chunks'][$chunkIndex])) {
    die(json_encode(['status' => 'already_uploaded']));
}

5.2 客户端处理续传

javascript复制async function checkUploadStatus(file) {
  const fileHash = await calculateMD5(file);
  const response = await fetch(`/status.php?hash=${fileHash}`);
  const status = await response.json();
  
  return status.chunks || [];
}

async function resumeUpload(file, existingChunks) {
  const chunkSize = 5 * 1024 * 1024;
  const totalChunks = Math.ceil(file.size / chunkSize);
  
  for (let i = 0; i < totalChunks; i++) {
    if (!existingChunks[i]) {
      await uploadChunk(file, i);
    }
  }
}

6. 安全防护措施

6.1 文件类型验证

php复制$allowedTypes = [
    'application/pdf' => 'pdf',
    'image/jpeg' => 'jpg',
    'image/png' => 'png'
];

$finfo = finfo_open(FILEINFO_MIME_TYPE);
$mime = finfo_file($finfo, $_FILES['chunk']['tmp_name']);
finfo_close($finfo);

if (!isset($allowedTypes[$mime])) {
    http_response_code(415);
    die("Unsupported file type");
}

6.2 病毒扫描集成

php复制function scanForViruses($filePath) {
    $clamscan = '/usr/bin/clamscan';
    if (file_exists($clamscan)) {
        exec("$clamscan --no-summary $filePath", $output, $return);
        return $return === 0;
    }
    return true; // 如果没有安装扫描器，默认通过
}

if (!scanForViruses($chunkPath)) {
    unlink($chunkPath);
    die("Virus detected in uploaded file");
}

7. 云存储集成方案

7.1 直接上传到S3

前端使用预签名URL：

javascript复制async function getPresignedUrl(fileName) {
  const response = await fetch('/s3_sign.php', {
    method: 'POST',
    body: JSON.stringify({ filename: fileName })
  });
  return await response.json();
}

async function uploadToS3(file) {
  const { url, fields } = await getPresignedUrl(file.name);
  const formData = new FormData();
  
  Object.entries(fields).forEach(([key, value]) => {
    formData.append(key, value);
  });
  
  formData.append('file', file);
  
  await fetch(url, {
    method: 'POST',
    body: formData
  });
}

PHP生成预签名URL：

php复制require 'vendor/autoload.php';
use Aws\S3\S3Client;

$s3 = new S3Client([
    'version' => 'latest',
    'region'  => 'us-east-1'
]);

$bucket = 'your-bucket';
$key = 'uploads/' . uniqid() . '_' . $_POST['filename'];

$cmd = $s3->getCommand('PutObject', [
    'Bucket' => $bucket,
    'Key'    => $key,
    'ACL'    => 'private'
]);

$request = $s3->createPresignedRequest($cmd, '+15 minutes');
$presignedUrl = (string)$request->getUri();

echo json_encode([
    'url' => $presignedUrl,
    'fields' => []
]);

8. 性能优化技巧

8.1 内存使用优化

php复制// 使用流处理代替内存操作
$in = fopen($_FILES['chunk']['tmp_name'], 'rb');
$out = fopen($targetPath, 'ab'); // a模式追加，b模式二进制

while ($data = fread($in, 8192)) {
    fwrite($out, $data);
}

fclose($in);
fclose($out);

8.2 并发上传控制

前端限制并发数：

javascript复制const MAX_CONCURRENT = 3;
let activeUploads = 0;
const queue = [];

async function processQueue() {
  while (queue.length > 0 && activeUploads < MAX_CONCURRENT) {
    activeUploads++;
    const { file, chunkIndex } = queue.shift();
    await uploadChunk(file, chunkIndex);
    activeUploads--;
    processQueue();
  }
}

function enqueueUpload(file, chunkIndex) {
  queue.push({ file, chunkIndex });
  processQueue();
}

9. 常见问题排查

9.1 上传中断问题

检查点：

1. Nginx/Apache的client_max_body_size或LimitRequestBody
2. PHP的post_max_size必须大于upload_max_filesize
3. 临时目录的inode限制：df -i
4. 防火墙或安全组限制

9.2 性能瓶颈分析

使用XHProf进行分析：

php复制xhprof_enable(XHPROF_FLAGS_CPU + XHPROF_FLAGS_MEMORY);

// 上传处理代码...

$xhprof_data = xhprof_disable();
include_once "/path/to/xhprof_lib/utils/xhprof_lib.php";
include_once "/path/to/xhprof_lib/utils/xhprof_runs.php";
$xhprof_runs = new XHProfRuns_Default();
$run_id = $xhprof_runs->save_run($xhprof_data, "upload");

10. 完整方案推荐

对于生产环境，建议采用以下组合方案：

1. 前端：

   • 使用Dropzone.js或Uppy实现分块上传
   • 添加MD5校验确保文件完整性

2. 后端：

   • Nginx + PHP-FPM配置优化
   • Redis记录上传状态
   • 最终存储到云存储（S3/OSS等）

3. 监控：

   • Prometheus监控上传成功率
   • ELK收集上传日志

示例架构：

code复制客户端 → (分块) → Nginx → PHP → 
    → 临时存储 → (合并) → 
    → 云存储/本地存储
    ↑
Redis记录状态