I'm trying to modify the open source library of cloudinary so that I can listen to the progress of the upload of my photo. The library class contains a MultipartUtility java class that I modified to listen to the progress of the upload.
The original code before modifications can be found on github: https://github.com/cloudinary/cloudinary_java/blob/master/cloudinary-android/src/main/java/com/cloudinary/android/MultipartUtility.java
I literally modified it to resemble the code from another cloud service CloudFS that supports progress when uploading files / images etc:
package com.cloudinary.android; import com.cloudinary.Cloudinary; import java.io.*; import java.net.HttpURLConnection; import java.net.URL; import java.util.Map; /** * This utility class provides an abstraction layer for sending multipart HTTP * POST requests to a web server. * * @author www.codejava.net * @author Cloudinary */ public class MultipartUtility { private final String boundary; private static final String LINE_FEED = "\r\n"; private static final String APPLICATION_OCTET_STREAM = "application/octet-stream"; private HttpURLConnection httpConn; private String charset; private OutputStream outputStream; private PrintWriter writer; UploadingCallback uploadingCallback; public final static String USER_AGENT = "CloudinaryAndroid/" + Cloudinary.VERSION; Long filesize; public void setUploadingCallback(UploadingCallback uploadingCallback) { this.uploadingCallback = uploadingCallback; } /** * This constructor initializes a new HTTP POST request with content type is * set to multipart/form-data * * @param requestURL * @param charset * @throws IOException */ public MultipartUtility(String requestURL, String charset, String boundary, Map<String, String> headers, Long filesize) throws IOException { this.charset = charset; this.boundary = boundary; this.filesize = filesize; URL url = new URL(requestURL); httpConn = (HttpURLConnection) url.openConnection(); httpConn.setDoOutput(true); // indicates POST method httpConn.setDoInput(true); httpConn.setFixedLengthStreamingMode(filesize); //added this in if (headers != null) { for (Map.Entry<String, String> header : headers.entrySet()) { httpConn.setRequestProperty(header.getKey(), header.getValue()); } } httpConn.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary); httpConn.setRequestProperty("User-Agent", USER_AGENT); outputStream = httpConn.getOutputStream(); writer = new PrintWriter(new OutputStreamWriter(outputStream, charset), true); } public MultipartUtility(String requestURL, String charset, String boundary) throws IOException { this(requestURL, charset, boundary, null, 0L); } /** * Adds a form field to the request * * @param name field name * @param value field value */ public void addFormField(String name, String value) { writer.append("--" + boundary).append(LINE_FEED); writer.append("Content-Disposition: form-data; name=\"" + name + "\"").append(LINE_FEED); writer.append("Content-Type: text/plain; charset=" + charset).append(LINE_FEED); writer.append(LINE_FEED); writer.append(value).append(LINE_FEED); writer.flush(); } /** * Adds a upload file section to the request * * @param fieldName name attribute in {@code <input type="file" name="..." />} * @param uploadFile a File to be uploaded * @throws IOException */ public void addFilePart(String fieldName, File uploadFile, String fileName) throws IOException { if (fileName == null) fileName = uploadFile.getName(); FileInputStream inputStream = new FileInputStream(uploadFile); addFilePart(fieldName, inputStream, fileName); } public void addFilePart(String fieldName, File uploadFile) throws IOException { addFilePart(fieldName, uploadFile, "file"); } public void addFilePart(String fieldName, InputStream inputStream, String fileName) throws IOException { if (fileName == null) fileName = "file"; writer.append("--" + boundary).append(LINE_FEED); writer.append("Content-Disposition: form-data; name=\"" + fieldName + "\"; filename=\"" + fileName + "\"").append(LINE_FEED); writer.append("Content-Type: ").append(APPLICATION_OCTET_STREAM).append(LINE_FEED); writer.append("Content-Transfer-Encoding: binary").append(LINE_FEED); writer.append(LINE_FEED); writer.flush(); int progress = 0; byte[] buffer = new byte[4096]; int bytesRead = -1; while ((bytesRead = inputStream.read(buffer)) != -1) { outputStream.write(buffer, 0, bytesRead); progress += bytesRead; /* int percentage = ((progress / filesize.intValue()) * 100);*/ if (uploadingCallback != null) { uploadingCallback.uploadListener(progress); } } outputStream.flush(); writer.flush(); uploadingCallback = null; inputStream.close(); writer.append(LINE_FEED); writer.flush(); } public void addFilePart(String fieldName, InputStream inputStream) throws IOException { addFilePart(fieldName, inputStream, "file"); } /** * Completes the request and receives response from the server. * * @return a list of Strings as response in case the server returned status * OK, otherwise an exception is thrown. * @throws IOException */ public HttpURLConnection execute() throws IOException { writer.append("--" + boundary + "--").append(LINE_FEED); writer.close(); return httpConn; } } The changes I made were to add on the following to the httpURLConnection as recommended by this thread: How to implement file upload progress bar in android: httpConn.setFixedLengthStreamingMode(filesize);
I then created a simple interface to listen for the upload progress:
public interface UploadingCallback { void uploadListener(int progress); } And then I attached it while the HttpURLConnection wrote the photo:
while ((bytesRead = inputStream.read(buffer)) != -1) { outputStream.write(buffer, 0, bytesRead); progress += bytesRead; /* int percentage = ((progress / filesize.intValue()) * 100);*/ if (uploadingCallback != null) { uploadingCallback.uploadListener(progress); } } The code ran but the progress of the upload doesn't seem to be measured correctly. The photo was about 365kb and the upload took about a 10th of a second (I started the upload at 17:56:55.481 and by 17:56:55.554 it was done, thats is just over 0.7 seconds). I do not believe my internet connection is that fast and expect it to take at least 5 seconds. I have a feeling it is measuring the time it took to write the photo to the buffer instead of the time it took to send it to cloudinary's servers.
How can I get it to measure the time it takes to upload the photo so that I can use the data for my progress bar?
04-24 17:56:55.481 28306-28725/com.a upload 4096 04-24 17:56:55.486 28306-28725/com.a upload 8192 04-24 17:56:55.486 28306-28725/com.a upload 12288 04-24 17:56:55.486 28306-28725/com.a upload 16384 04-24 17:56:55.487 28306-28725/com.a upload 20480 04-24 17:56:55.487 28306-28725/com.a upload 24576 04-24 17:56:55.487 28306-28725/com.a upload 28672 04-24 17:56:55.487 28306-28725/com.a upload 32768 04-24 17:56:55.491 28306-28725/com.a upload 36864 04-24 17:56:55.492 28306-28725/com.a upload 40960 04-24 17:56:55.493 28306-28725/com.a upload 45056 04-24 17:56:55.493 28306-28725/com.a upload 49152 04-24 17:56:55.493 28306-28725/com.a upload 53248 04-24 17:56:55.493 28306-28725/com.a upload 57344 04-24 17:56:55.494 28306-28725/com.a upload 61440 04-24 17:56:55.494 28306-28725/com.a upload 65536 04-24 17:56:55.494 28306-28725/com.a upload 69632 04-24 17:56:55.494 28306-28725/com.a upload 73728 04-24 17:56:55.494 28306-28725/com.a upload 77824 04-24 17:56:55.495 28306-28725/com.a upload 81920 04-24 17:56:55.495 28306-28725/com.a upload 86016 04-24 17:56:55.495 28306-28725/com.a upload 90112 04-24 17:56:55.495 28306-28725/com.a upload 94208 04-24 17:56:55.495 28306-28725/com.a upload 98304 04-24 17:56:55.495 28306-28725/com.a upload 102400 04-24 17:56:55.495 28306-28725/com.a upload 106496 04-24 17:56:55.496 28306-28725/com.a upload 110592 04-24 17:56:55.496 28306-28725/com.a upload 114688 04-24 17:56:55.496 28306-28725/com.a upload 118784 04-24 17:56:55.497 28306-28725/com.a upload 122880 04-24 17:56:55.498 28306-28725/com.a upload 126976 04-24 17:56:55.498 28306-28725/com.a upload 131072 04-24 17:56:55.498 28306-28725/com.a upload 135168 04-24 17:56:55.498 28306-28725/com.a upload 139264 04-24 17:56:55.499 28306-28725/com.a upload 143360 04-24 17:56:55.506 28306-28725/com.a upload 147456 04-24 17:56:55.510 28306-28725/com.a upload 151552 04-24 17:56:55.510 28306-28725/com.a upload 155648 04-24 17:56:55.514 28306-28725/com.a upload 159744 04-24 17:56:55.515 28306-28725/com.a upload 163840 04-24 17:56:55.517 28306-28725/com.a upload 167936 04-24 17:56:55.517 28306-28725/com.a upload 172032 04-24 17:56:55.518 28306-28725/com.a upload 176128 04-24 17:56:55.518 28306-28725/com.a upload 180224 04-24 17:56:55.518 28306-28725/com.a upload 184320 04-24 17:56:55.519 28306-28725/com.a upload 188416 04-24 17:56:55.519 28306-28725/com.a upload 192512 04-24 17:56:55.519 28306-28725/com.a upload 196608 04-24 17:56:55.519 28306-28725/com.a upload 200704 04-24 17:56:55.520 28306-28725/com.a upload 204800 04-24 17:56:55.525 28306-28725/com.a upload 208896 04-24 17:56:55.526 28306-28725/com.a upload 212992 04-24 17:56:55.527 28306-28725/com.a upload 217088 04-24 17:56:55.530 28306-28725/com.a upload 221184 04-24 17:56:55.530 28306-28725/com.a upload 225280 04-24 17:56:55.530 28306-28725/com.a upload 229376 04-24 17:56:55.530 28306-28725/com.a upload 233472 04-24 17:56:55.530 28306-28725/com.a upload 237568 04-24 17:56:55.531 28306-28725/com.a upload 241664 04-24 17:56:55.532 28306-28725/com.a upload 245760 04-24 17:56:55.532 28306-28725/com.a upload 249856 04-24 17:56:55.532 28306-28725/com.a upload 253952 04-24 17:56:55.533 28306-28725/com.a upload 258048 04-24 17:56:55.533 28306-28725/com.a upload 262144 04-24 17:56:55.535 28306-28725/com.a upload 266240 04-24 17:56:55.540 28306-28725/com.a upload 270336 04-24 17:56:55.540 28306-28725/com.a upload 274432 04-24 17:56:55.541 28306-28725/com.a upload 278528 04-24 17:56:55.541 28306-28725/com.a upload 282624 04-24 17:56:55.543 28306-28725/com.a upload 286720 04-24 17:56:55.545 28306-28725/com.a upload 290816 04-24 17:56:55.545 28306-28725/com.a upload 294912 04-24 17:56:55.547 28306-28725/com.a upload 299008 04-24 17:56:55.547 28306-28725/com.a upload 303104 04-24 17:56:55.547 28306-28725/com.a upload 307200 04-24 17:56:55.547 28306-28725/com.a upload 311296 04-24 17:56:55.547 28306-28725/com.a upload 315392 04-24 17:56:55.548 28306-28725/com.a upload 319488 04-24 17:56:55.548 28306-28725/com.a upload 323584 04-24 17:56:55.548 28306-28725/com.a upload 327680 04-24 17:56:55.548 28306-28725/com.a upload 331776 04-24 17:56:55.549 28306-28725/com.a upload 335872 04-24 17:56:55.549 28306-28725/com.a upload 339968 04-24 17:56:55.549 28306-28725/com.a upload 344064 04-24 17:56:55.550 28306-28725/com.a upload 348160 04-24 17:56:55.550 28306-28725/com.a upload 352256 04-24 17:56:55.551 28306-28725/com.a upload 356352 04-24 17:56:55.551 28306-28725/com.a upload 360448 04-24 17:56:55.552 28306-28725/com.a upload 364544 04-24 17:56:55.554 28306-28725/com.a upload 365790 To test this out for yourself, you will need to create a free account on cloudinary website in order to get your cloudname so you can connect your Android SDK to their services for an unsigned direct upload from android directly to their servers.
EDIT:
This is what I have tried and it still jumps from 0 - 100% in 0.7 seconds when the upload actually finishes in 7 seconds time:
while ((bytesRead = inputStream.read(buffer)) != -1) { outputStream.write(buffer, 0, bytesRead); progress += bytesRead; Log.d("MultiPart", "file transferred so far: " + progress); if (uploadingCallback != null) { uploadingCallback.uploadListener(progress); } Log.d("Flushing", "flush the writer"); outputStream.flush(); writer.flush(); } 2 Answers
Answers 1
There is a problem in the use of flush() method and the time you call update callback().
As you can see from your code every time you read part of the picture, you write it to the output buffer, but that does not mean it's sent to the server, it might be buffered, and then later on write'n to the server.
You have two options, either call outputStream.flush() after every outputStream.write(), but that will kill the performance of the upload, because you would lose the benefits of buffering.
Or you could call your updateCallback() after the outputStream.flush() at the end of your method. Because after outputStream.flush() you are certain that the data is on the server, and that progress is over.
For more info about the flush see this thread What is the purpose of flush() in Java streams?
Best regards.
Answers 2
This is a shot in the dark because I have not tested on an Android environment, however I would recommend trying the following.
Instead of using a fixed length use setChunkedStreamingMode
//httpConn.setFixedLengthStreamingMode(filesize); httpConn.setChunkedStreamingMode(4096); // or whatever size you see fit doing this should trigger part of the request to get sent every time you send in 4096 bytes of data and essentially flushing out the internal buffer.
You could also try manually flushing the buffer after each write, this could slow down the file upload especially if you flush to often however it would likely fix your problem. You might end up playing with buffer sizes to find a sweet spot.
while ((bytesRead = inputStream.read(buffer)) != -1) { outputStream.write(buffer, 0, bytesRead); progress += bytesRead; /* int percentage = ((progress / filesize.intValue()) * 100);*/ if (uploadingCallback != null) { uploadingCallback.uploadListener(progress); } // trigger the stream to write its data outputStream.flush(); } With either of these changes you would likely want to let the user choose to set their own buffer size instead of passing in the total file size. EG change your constructor to the following:
MultipartUtility(String requestURL, String charset, String boundary, Map<String, String> headers, int chunkSize) 
0 comments:
Post a Comment