多线程传输实现

实现原理

将源文件按长度为分为N块文件,然后开辟N个线程,每个线程传输一块,最后合并所有线线程文件.比如
一个文件500M我们按长度可以分5个线程传输.第一线程从0-100M,第二线程从100M-200M......最后合并5个线程文件.

实现流程

1.客户端向服务端请求文件信息(名称,长度)
2.客户端跟据文件长度开辟N个线程连接服务端
3.服务端开辟新的线程与客户端通信并传输文件
4.客户端将每线程数据保存到一个文件
5.合并所有线程文件

编码实现

大体说来就是按以上步骤进行,详细的实现和一些要点,我们跟据以上流程在编码中实现

结构定义

在通信过程中需要传递的信息包括文件名称,文件长度,文件偏移,操作指令等信息,为了方便操作我们定义如下结构
复制内容到剪贴板代码:
typedef struct
{
        char        Name[100];        //文件名称
        int                FileLen;        //文件长度
        int                CMD;                //操作指令
        int                seek;                //线程开始位置
        SOCKET  sockid;               
}FILEINFO;
1.请求文件信息

客户端代码如下
复制内容到剪贴板代码:
        FILEINFO fi;
        memset((char*)&fi,0,sizeof(fi));
        fi.CMD=1;                //得到文件信息

        if(send(client,(char*)&fi,sizeof(fi),0)==SOCKET_ERROR)
        {
                cout<<"Send  Get FileInfo Error\n";
        }
服务端代码如下

        while(true)
        {
                SOCKET client;
                if(client=accept(server,(sockaddr *)&clientaddr,&len))
                {
                        FILEINFO RecvFileInfo;
                        memset((char*)&RecvFileInfo,0,sizeof(RecvFileInfo));
                        if(recv(client,(char*)&RecvFileInfo,sizeof(RecvFileInfo),0)==SOCKET_ERROR)
                        {
                                cout<<"The Clinet Socket is Closed\n";
                                break;
                        }else
                        {
                                EnterCriticalSection(&CS);                //进入临界区
                                memcpy((char*)&TempFileInfo,(char*)&RecvFileInfo,sizeof(RecvFileInfo));
                                switch(TempFileInfo.CMD)
                                        {
                                        case 1:
                                                 GetInfoProc        (client);
                                                 break;
                                        case 2:
                                                 TempFileInfo.sockid=client;
                                                 CreateThread(NULL,NULL,GetFileProc,NULL,NULL,NULL);
                                                 break;
                                        }
                                LeaveCriticalSection(&CS);                //离开临界区
                        }
                }
        }
在这里服务端循环接受连接,并跟据TempFileInfo.CMD来判断客户端的请求类型,1为请求文件信息,2为下载文件
因为在下载文件的请求中,需要开辟新的线程,并传递文件偏移和文件大小等信息,所以需要对线程同步.这里使用临界区
其文件信息函数GetInfoProc代码如下
复制内容到剪贴板代码:
DWORD GetInfoProc(SOCKET client)
{
        CFile        file;
        if(file.Open(FileName,CFile::modeRead|CFile::typeBinary))
        {
                int FileLen=file.GetLength();
                if(send(client,(char*)&FileLen,sizeof(FileLen),0)==SOCKET_ERROR)
                {
                        cout<< "Send FileLen Error\n";
                }else
                {
                        cout<< "The Filelen is "<<FileLen<<"\n\n";
                }
        }
        return 0;
}
这里主要是向客户端传递文件长度,而客户端收到长度后则开辟线程进行连接传输文件

2.客户端跟据长度开辟线程

其实现代码如下
复制内容到剪贴板代码:
        FILEINFO FI;
        int FileLen=0;
        if(recv(client,(char*)&FileLen,sizeof(FileLen),0)==SOCKET_ERROR)//接受文件长度
        {
                cout<<"Recv FileLen Error\n";
        }else
        {
                cout<<"FileLen is "<<FileLen<<"\n";
                int COUNT_SIZE=FileLen/5;                        //每线程传输大小                       
                for(int i=0;i<5;i++)                                //分5个线程传输
                {
                        EnterCriticalSection(&CS);                //进入临界区
                        memset((char*)&FI,0,sizeof(FI));
                        FI.CMD=2;                                //请求下载文件
                        FI.seek=i*COUNT_SIZE;                        //线程文件偏移
                        if(i+1==5)                                //最后一线程长度为总长度减前4个线程长度
                        {
                                FI.FileLen=FileLen-COUNT_SIZE*i;
                        }else
                        {
                                FI.FileLen=COUNT_SIZE;
                        }
                        Thread[i]=CreateThread(NULL,NULL,GetFileThread,&i,NULL,NULL);
                        Sleep(500);
                        LeaveCriticalSection(&CS);                //离开临界区
                }
        }
        WaitForMultipleObjects(5,Thread,true,INFINITE);                //等所有线程结束
这里默认开辟5个线程传输,当然可以改为想要的线程数目,仍然用临界区来实现线程的同步问题

3.服务端开辟线程传输数据

在1.请求文件信息中以说明了服务端的结构,这里主要介绍线程函数的实现,其代码如下
复制内容到剪贴板代码:
DWORD WINAPI GetFileProc(LPVOID lparam)
{
        EnterCriticalSection(&CS);                        //进入临界区
        int FileLen=TempFileInfo.FileLen;
        int Seek=TempFileInfo.seek;
        SOCKET client=TempFileInfo.sockid;
        LeaveCriticalSection(&CS);                        //离开临界区

        CFile        file;
        if(file.Open(FileName,CFile::modeRead|CFile::typeBinary))
        {
                file.Seek(Seek,CFile::begin);                //指针移至偏移位置
                char *date=new char[FileLen];
                int nLeft=FileLen;
                int idx=0;
                file.Read(date,FileLen);
                while(nLeft>0)
                {
                        int ret=send(client,&date[idx],nLeft,0);
                        if(ret==SOCKET_ERROR)
                        {
                                cout<<"Send Date Error \n";
                                break;
                        }
                        nLeft-=ret;
                        idx+=ret;
                }
                file.Close();
                delete[] date;
        }else
        {
                cout<<"open the file error\n";
        }
        closesocket(client);
        return 0;
}
还是比较简单的,主要是获取线程的文件长度和偏移,并移动文件指针到偏移处,最后读取发送数据,而客户端
接受数据并写入文件.

4.客户端将线程数据保存到文件

GetFileThread的实现代码如下
复制内容到剪贴板代码:
DWORD WINAPI GetFileThread(LPVOID lparam)
{
        char TempName[MAX_PATH];
        sprintf(TempName,"TempFile%d",*(DWORD*)lparam);        //每线程的文件名为"TempName"+线程数
        SOCKET client;
        SOCKADDR_IN serveraddr;
        int port=5555;
        client=socket(AF_INET,SOCK_STREAM,IPPROTO_TCP);
        serveraddr.sin_family=AF_INET;
        serveraddr.sin_port=htons(port);
        serveraddr.sin_addr.S_un.S_addr=inet_addr("127.0.0.1");
        if(connect(client,(SOCKADDR*)&serveraddr,sizeof(serveraddr))==INVALID_SOCKET)
        {
                cout<<"Connect Server Error\n";
        }
        EnterCriticalSection(&CS);                //进入临界区
        if(send(client,(char*)&FI,sizeof(FI),0)==SOCKET_ERROR)
        {
                cout<<"Send  GetFile Error\n";
                return 0;
        }
        CFile        file;
        int FileLen=FI.FileLen;                        //文件长度
        int Seek=FI.seek;                        //文件偏移
        LeaveCriticalSection(&CS);                //离开临界区

        if(file.Open(TempName,CFile::modeWrite|CFile::typeBinary|CFile::modeCreate))
        {
                char *date = new char[FileLen];
                int nLeft=FileLen;
                int idx=0;
                while(nLeft>0)
                {
                        int ret=recv(client,&date[idx],nLeft,0);
                        if(ret==SOCKET_ERROR)
                        {
                                cout<<"Recv Date Error";
                                break;
                        }
                        idx+=ret;
                        nLeft-=ret;
                }
                file.Write(date,FileLen);
                file.Close();
                delete[] date;
        }else
        {
                cout<<"Create File Error\n";
        }
        return 0;
}
在此线程函数中,将每线程传输的数据存为一个文件,文件名为"TempName"+线程数,只所以存成单独的文件是
因为比较直观且容易理解,但如果文件很大的话这个方法并不好,因为合并文件又会花费很多时间,另一个方法
是 创始一个文件,让每个线程写入文件的不同偏移,这样就可以不必单独合并文件了,但要记得打开文件时
加入CFile::shareDenyNone属性.这样整个过程就完成了.最后一步合并线程文件

5.合并线程文件
复制内容到剪贴板代码:
int UniteFile()                //合并线程文件
{
        cout<<"Now is Unite Fileing...\n";
        int                len;
        char        *date;
        CFile        file;
        CFile        file0;
        /*其它文件......*/

        if(file.Open(FileName,CFile::modeCreate|CFile::typeBinary|CFile::modeWrite))//创建文件
        {
                file0.Open("TempFile0",CFile::modeRead|CFile::typeBinary);//合并第一线程文件
                len=file0.GetLength();
                date=new char[len];
                file0.Read(date,len);
                file.SeekToEnd();
                file.Write(date,len);
                file1.Open("TempFile1",CFile::modeRead|CFile::typeBinary);//合并第二线程文件
                len=file1.GetLength();
                date=new char[len];
                file1.Read(date,len);
                file.SeekToEnd();
                file.Write(date,len);
                /*合并其它线程......*/

               
                file0.Close();
                file1.Close();
                /*.......*/
                delete[] date;
               
                return true;
        }else
        {
                return false;
        }

}
这个简单,就是打开一个文件读取到缓冲区,写入文件,再打开第二个......现在多线程传输部分就介绍完了
下面讨论断断点续传的实现

断点续传

所谓的断点续传就是指:文件在传输过程式中被中断后,在重新传输时,可以从上次的断点处开始传输,这样就可
节省时间,和其它资源.

实现关键

在这里有两个关键点,其一是检测本地已经下载的文件长度和断点值,其二是在服务端调整文件指针到断点处

实现方法

我们用一个简单的方法来实现断点续传的功能.在传输文件的时候创建一个临时文件用来存放文件的断点位置
在每次发送接受文件时,先检查有没有临时文件,如果有的话就从临时文件中读取断点值,并把文件指针移动到
断点位置开始传输,这样便可以做到断点续传了

实现流程

首次传输其流程如下

1.服务端向客户端传递文件名称和文件长度
2.跟据文件长度计算文件块数(文件分块传输请参照第二篇文章)
3.客户端将传输的块数写入临时文件(做为断点值)
4.若文件传输成功则删除临时文件

首次传输失败后将按以下流程进行

1.客户端从临时文件读取断点值并发送给服务端
2.服务端与客户端将文件指针移至断点处
3.从断点处传输文件

编码实现

因为程序代码并不复杂,且注释也比较详细,这里就给出完整的实现

其服务端实现代码如下
复制内容到剪贴板代码:
int _tmain(int argc, TCHAR* argv[], TCHAR* envp[])
{
        cout<<"\t\t服务端-断点续传"<<"\t  作者:冷风\n\n"<<"请输入被下载的文件路径 如 C:\\File.rar \n\n"<<"文件路径:  ";
        cin        >>FilePath;
        /*这部分为网络参数与设置,详细请参照源代码......*/
        while(true)
        {
                if(client=accept(server,(sockaddr *)&clientaddr,&len))
                {
                        cout<<"have one connect\n";
                        int nCurrentPos=0;//接受断点值
                        if(recv(client,(char*)&nCurrentPos,sizeof(nCurrentPos),0)==SOCKET_ERROR)
                        {
                                cout<<"The Clinet Socket is Closed\n";
                                break;
                        }else
                        {
                                cout<<"The Currentpos is The"<<nCurrentPos<<"\n";
                                GetFileProc        (nCurrentPos,client);
                        }
                }
        }
        closesocket(server);
        closesocket(client);
        WSACleanup();
        return 0;
        return 0;
}

DWORD  GetFileProc        (int nCurrentPos,SOCKET client)
{
        cout <<"Get File Proc is ok\n";
        CFile        file;
        int                nChunkCount=0;        //文件块数
        if(file.Open(FilePath,CFile::modeRead|CFile::typeBinary))
        {
                if(nCurrentPos!=0)
                {
                        file.Seek(nCurrentPos*CHUNK_SIZE,CFile::begin);        //文件指针移至断点处
                        cout<<"file seek is "<<nCurrentPos*CHUNK_SIZE<<"\n";
                }
                int FileLen=file.GetLength();
                nChunkCount=FileLen/CHUNK_SIZE;                                //文件块数
                if(FileLen%nChunkCount!=0)
                        nChunkCount++;
                send(client,(char*)&FileLen,sizeof(FileLen),0);                //发送文件长度
                char *date=new char[CHUNK_SIZE];
                for(int i=nCurrentPos;i<nChunkCount;i++)                //从断点处分块发送
                {       
                        cout<<"send the count"<<i<<"\n";
                        int nLeft;
                        if(i+1==nChunkCount)                                //最后一块
                                nLeft=FileLen-CHUNK_SIZE*(nChunkCount-1);
                        else
                                nLeft=CHUNK_SIZE;
                        int idx=0;
                        file.Read(date,CHUNK_SIZE);
                        while(nLeft>0)
                        {
                                int ret=send(client,&date[idx],nLeft,0);
                                if(ret==SOCKET_ERROR)
                                {
                                        cout<<"Send The Date Error \n";
                                        break;
                                }
                                nLeft-=ret;
                                idx+=ret;
                        }
                }
                file.Close();
                delete[] date;
        }else
        {
                cout<<"open the file error\n";
        }
        return 0;
}
客户端实现代码如下
复制内容到剪贴板代码:
int _tmain(int argc, TCHAR* argv[], TCHAR* envp[])
{
        cout<<"\t\t客户端-断点续传"<<"\t  作者:冷风\n\n"<<"请输入保存文件的路径 如 C:\\Save.RAR \n\n"<<"文件路径:  ";
        cin        >>FilePath;
        /*网络参数初示化,详细请参照源代码......*/
        if(connect(client,(SOCKADDR*)&serveraddr,sizeof(serveraddr))==INVALID_SOCKET)
        {
                cout<<"Connect Server Error";
                return 0;
        }
        int FileLen=0;

        int nCurrentPos=0; //断点位置

        UINT OpenFlags;       

        CFile PosFile;
        if(PosFile.Open("PosFile.temp",CFile::modeRead|CFile::typeBinary))//如果有临时文件则读取断点
        {
                PosFile.Read((char*)&nCurrentPos,sizeof(nCurrentPos));        //读取断点位置
                cout<<"The File Pos is "<<nCurrentPos<<"\n";
                nCurrentPos=nCurrentPos+1;                                //从断点的下一块开始
                PosFile.Close();
                send(client,(char*)&nCurrentPos,sizeof(nCurrentPos),0);        //发送断点值
                OpenFlags=CFile::modeWrite|CFile::typeBinary;                //文件为可写
        }
        else
        {
                send(client,(char*)&nCurrentPos,sizeof(nCurrentPos),0);        //无断点文件nCurrentPos为0
                OpenFlags=CFile::modeWrite|CFile::typeBinary|CFile::modeCreate;//创建文件方式
        }

        if(recv(client,(char*)&FileLen,sizeof(FileLen),0)!=0)//接受文件长度
        {
                int                nChunkCount;
                CFile        file;
                nChunkCount=FileLen/CHUNK_SIZE;                //计算文件块数
                if(FileLen%nChunkCount!=0)
                {
                        nChunkCount++;
                }

                if(file.Open(FilePath,OpenFlags))
                {
                        file.Seek(nCurrentPos*CHUNK_SIZE,CFile::begin);        //文件指针移至断点处

                        char *date = new char[CHUNK_SIZE];

                        for(int i=nCurrentPos;i<nChunkCount;i++)        //从断点处开始写入文件
                        {
                                cout<<"Recv The Chunk is "<<i<<"\n";
                                int nLeft;
                                if(i+1==nChunkCount)                                                //最后一块
                                        nLeft=FileLen-CHUNK_SIZE*(nChunkCount-1);
                                else
                                        nLeft=CHUNK_SIZE;
                                int idx=0;
                                while(nLeft>0)
                                {
                                        int ret=recv(client,&date[idx],nLeft,0);
                                        if(ret==SOCKET_ERROR)
                                        {
                                                cout<<"Recv Date Error";
                                                return 0;
                                        }
                                        idx+=ret;
                                        nLeft-=ret;
                                }
                                file.Write(date,CHUNK_SIZE);
                                CFile        PosFile;                //将断点写入PosFile.temp文件
                                int seekpos=i+1;
                                if(PosFile.Open("PosFile.temp",CFile::modeWrite|CFile::typeBinary|CFile::modeCreate));
                                {
                                        PosFile.Write((char*)&seekpos,sizeof(seekpos));
                                        PosFile.Close();
                                }
                        }
                        file.Close();
                        delete[] date;
                }       
                if(DeleteFile("PosFile.temp")!=0)
                {
                        cout<<"文件传输完成";
                }
        }
        return 0;
}
客户端运行时会试图打开临时文件,如果存在则读取断点值,如果不存在则断点为0,打开文件后将文件指针移至
断点处开始接受数据,每接受一块就把当前块的数值存入临时文件.其实现代码比较简单结合上面的流程介绍
看代码应该没什么难度,所以我也就不画蛇添足了.

到此文件传输部分就介绍完毕了,在写文件传输这一系列程序的过程中
界面实现主要参考了VC知识库王景生的<<VC控件TreeCtrl与ListCtrl演示>>一文
在功能实现一篇中主要参考了"草草"的SEU_PEEPER木马的源代码(强烈推荐一下,草草如果看到了一定要请客哦)
而在本篇中主要参考了 2005电脑报 王育文<<简单文件传输实现>>一文,如果有问题的话参考一下上面的著作
相信会有很大收获的,当然更欢迎到黑或我的BLOG(Http:// blog.csdn.net/chinafe)上讨论