Recently I wrote some code (client and server) to send an image - the client simply uploads the image to the server, just using the socket module: Sending image over sockets (ONLY) in Python, image can not be open.
However, the image sending part is now what I am concerned with. This is the original image I'm using:
In my server code (which receives the images), I have these lines:
myfile = open(basename % imgcounter, 'wb') myfile.write(data) data = sock.recv(40960000) if not data: myfile.close() break myfile.write(data) myfile.close() sock.sendall("GOT IMAGE") sock.shutdown() But I don't think this is the best way of doing it. I think I should instead implement the server such that it receives the data in chunks:
#!/usr/bin/env python import random import socket, select from time import gmtime, strftime from random import randint imgcounter = 1 basename = "image%s.png" HOST = '127.0.0.1' PORT = 2905 connected_clients_sockets = [] server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) server_socket.bind((HOST, PORT)) server_socket.listen(10) connected_clients_sockets.append(server_socket) while True: read_sockets, write_sockets, error_sockets = select.select(connected_clients_sockets, [], []) for sock in read_sockets: if sock == server_socket: sockfd, client_address = server_socket.accept() connected_clients_sockets.append(sockfd) else: try: data = sock.recv(4096) txt = str(data) if data: if data.startswith('SIZE'): tmp = txt.split() size = int(tmp[1]) print 'got size %s' % size sock.sendall("GOT SIZE") elif data.startswith('BYE'): sock.shutdown() else : myfile = open(basename % imgcounter, 'wb') myfile.write(data) amount_received = 0 while amount_received < size: data = sock.recv(4096) amount_received += len(data) print amount_received if not data: break myfile.write(data) myfile.close() sock.sendall("GOT IMAGE") sock.shutdown() except: sock.close() connected_clients_sockets.remove(sock) continue imgcounter += 1 server_socket.close() But when I do this, the server prints:
got size 54674 4096 8192 12288 16384 20480 24576 28672 32768 36864 40960 45056 49152 50578 And then seems to hang, and the client hangs too. However, at the server's side I can see only a piece of the image the client wanted to send:
It seems like there are some bytes missing. What is the proper way of sending a huge amount of data (images, other type of file) using ONLY sockets?
2 Answers
Answers 1
I'm assuming that you have a particular reason for doing this with naked sockets, such as self-edification, which means that I won't answer by saying "You accidentally forgot to just use HTTP and Twisted", which perhaps you've heard before :-P. But really you should look at higher-level libraries at some point as they're a lot easier!
Define a protocol
If all you want is to send an image, then it can be simple:
Client -> server: 8 bytes: big endian, length of image.Client -> server: length bytes: all image data.- (
Client <- server: 1 byte, value 0: indicate transmission received - optional step you may not care if you're using TCP and just assume that it's reliable.)
Code it
server.py
import os from socket import * from struct import unpack class ServerProtocol: def __init__(self): self.socket = None self.output_dir = '.' self.file_num = 1 def listen(self, server_ip, server_port): self.socket = socket(AF_INET, SOCK_STREAM) self.socket.bind((server_ip, server_port)) self.socket.listen(1) def handle_images(self): try: while True: (connection, addr) = self.socket.accept() try: bs = connection.recv(8) (length,) = unpack('>Q', bs) data = b'' while len(data) < length: # doing it in batches is generally better than trying # to do it all in one go, so I believe. to_read = length - len(data) data += connection.recv( 4096 if to_read > 4096 else to_read) # send our 0 ack assert len(b'\00') == 1 connection.sendall(b'\00') finally: connection.shutdown(SHUT_WR) connection.close() with open(os.path.join( self.output_dir, '%06d.jpg' % self.file_num), 'w' ) as fp: fp.write(data) self.file_num += 1 finally: self.close() def close(self): self.socket.close() self.socket = None # could handle a bad ack here, but we'll assume it's fine. if __name__ == '__main__': sp = ServerProtocol() sp.listen('127.0.0.1', 55555) sp.handle_images() client.py
from socket import * from struct import pack class ClientProtocol: def __init__(self): self.socket = None def connect(self, server_ip, server_port): self.socket = socket(AF_INET, SOCK_STREAM) self.socket.connect((server_ip, server_port)) def close(self): self.socket.shutdown(SHUT_WR) self.socket.close() self.socket = None def send_image(self, image_data): # use struct to make sure we have a consistent endianness on the length length = pack('>Q', len(image_data)) # sendall to make sure it blocks if there's back-pressure on the socket self.socket.sendall(length) self.socket.sendall(image_data) ack = self.socket.recv(1) # could handle a bad ack here, but we'll assume it's fine. if __name__ == '__main__': cp = ClientProtocol() image_data = None with open('IMG_0077.jpg', 'r') as fp: image_data = fp.read() assert(len(image_data)) cp.connect('127.0.0.1', 55555) cp.send_image(image_data) cp.close() Answers 2
The problem is you are not incrementing amount_received for the first chunk of the data received.
Fix below:
#!/usr/bin/env python import random import socket, select from time import gmtime, strftime from random import randint imgcounter = 1 basename = "image%s.png" HOST = '127.0.0.1' PORT = 2905 connected_clients_sockets = [] server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) server_socket.bind((HOST, PORT)) server_socket.listen(10) connected_clients_sockets.append(server_socket) while True: read_sockets, write_sockets, error_sockets = select.select(connected_clients_sockets, [], []) for sock in read_sockets: if sock == server_socket: sockfd, client_address = server_socket.accept() connected_clients_sockets.append(sockfd) else: try: data = sock.recv(4096) txt = str(data) if data: if data.startswith('SIZE'): tmp = txt.split() size = int(tmp[1]) print 'got size %s' % size sock.sendall("GOT SIZE") elif data.startswith('BYE'): sock.shutdown() else : myfile = open(basename % imgcounter, 'wb') myfile.write(data) amount_received = len(data) # The fix! while amount_received < size: data = sock.recv(4096) amount_received += len(data) print amount_received if not data: break myfile.write(data) myfile.close() sock.sendall("GOT IMAGE") sock.shutdown() except: sock.close() connected_clients_sockets.remove(sock) continue imgcounter += 1 server_socket.close() 
0 comments:
Post a Comment