Uploading a file with progress in Kotlin

Using Retrofit, OkHttp, Okio, rxjava3

All source code : https://gist.github.com/VonLisboa/8a0fcafa07ccbf7c4337ef5ebcc27832

Do you want to upload a file using the clean Retrofit syntax, but aren’t sure how to receive the result as well as the upload progress? We will be using Retrofit to perform the file upload, building an implementation that is able to receive the completion progress at intervals and then complete with the remote API response.

Whilst long-running operations are happening, it is nice for the user to see that activity is occurring, such as a progress view being displayed. For the case of a file upload we can show the real progress, which can be represented by the number of bytes transmitted out of the total file size.

We will use the APIs available to us in Retrofit, OkHttp and Okio to build a class that can be used whenever we want a request to publish its progress to whoever wishes to observe it! 🆙

Before we continue: Please check out the article on my blog, Lord Codes, you will find code snippets with themed syntax highlighting and much more, it is definitely my preferred way to read it! 👍

Endpoint

We are developing a messaging application that is able to attach a file to a message thread. It is worth noting that the reactive component uses RxJava, however, it can be altered to use regular callbacks or Kotlin Coroutines and suspend functions.

Our endpoint is a POST request that contains a multipart body, consisting of the filename, file MIME type, file size and the file itself. We can define it using Retrofit, specifying the required parts.

package com.example.retrofit.upload

import io.reactivex.rxjava3.core.Single
import okhttp3.MultipartBody
import okhttp3.RequestBody
import retrofit2.http.Multipart
import retrofit2.http.POST
import retrofit2.http.Part


interface AttachmentRemoteApi {
@Multipart
@POST("file_uploader")
fun attachFile(
@Part("name") filename: RequestBody,
@Part("type") mimeType: RequestBody,
@Part("size") fileSize: RequestBody,
@Part files: MultipartBody.Part
): Single<AttachmentUploadedRemoteDto>
}

Counting progress

If we just wanted to upload the file without any progress, we would simply convert the file to a request body and send it in the request.

fun createUploadRequestBody(file: File, mimeType: String) =  file.asRequestBody(mimeType.toMediaType())

Monitoring upload progress can be achieved by using our own CountingRequestBody which wraps around the file RequestBody that would have been used before. The data that is transmitted is the same as before, allowing the raw file RequestBody to be delegated to for the content type and content length.

package com.example.retrofit.uploadimport okhttp3.RequestBody
import okio.BufferedSink
import okio.buffer
import java.io.IOException
class CountingRequestBody(
private val requestBody: RequestBody,
private val onProgressUpdate: CountingRequestListener
) : RequestBody() {
override fun contentType() = requestBody.contentType()
@Throws(IOException::class)
override fun contentLength() = requestBody.contentLength()
@Throws(IOException::class)
override fun writeTo(sink: BufferedSink) {
val countingSink = CountingSink(sink, this, onProgressUpdate)
val bufferedSink = countingSink.buffer()
requestBody.writeTo(bufferedSink)
bufferedSink.flush()
}
}

Transmitting the request body is performed by writing it to a Sink, we will wrap the default sink with our own one that counts the bytes that are transmitted and reports them back via a progress callback.

package com.example.retrofit.uploadimport okhttp3.RequestBody
import okio.Buffer
import okio.ForwardingSink
import okio.Sink
typealias CountingRequestListener = (bytesWritten: Long, contentLength: Long) -> Unitclass CountingSink(
sink: Sink,
private val requestBody: RequestBody,
private val onProgressUpdate: CountingRequestListener
) : ForwardingSink(sink) {
private var bytesWritten = 0L
override fun write(source: Buffer, byteCount: Long) {
super.write(source, byteCount)
bytesWritten += byteCount
onProgressUpdate(bytesWritten, requestBody.contentLength())
}
}

Within CountingRequestBody we can wrap the default sink into our new CountingSink and write to a buffered version of that, in order to both transmit the file and observe its progress. 👀

package com.example.retrofit.uploadimport okhttp3.RequestBody
import okio.BufferedSink
import okio.buffer
import java.io.IOException
class CountingRequestBody(
private val requestBody: RequestBody,
private val onProgressUpdate: CountingRequestListener
) : RequestBody() {
override fun contentType() = requestBody.contentType()
@Throws(IOException::class)
override fun contentLength() = requestBody.contentLength()
@Throws(IOException::class)
override fun writeTo(sink: BufferedSink) {
val countingSink = CountingSink(sink, this, onProgressUpdate)
val bufferedSink = countingSink.buffer()
requestBody.writeTo(bufferedSink)
bufferedSink.flush()
}
}

The result

Whilst observing the upload progress, there will either be progress or a completed response, the perfect candidate for a sealed class. This will allow CountingRequestResult to be the return type and callers can handle both progress updates and the completed result.

package com.example.retrofit.uploadsealed class CountingRequestResult<ResultT> {
data class Progress<ResultT>(
val progressFraction: Double
) : CountingRequestResult<ResultT>()
data class Completed<ResultT>(
val result: ResultT
) : CountingRequestResult<ResultT>()
}

Perform the upload

Now that we have a way of uploading a file and receiving the upload progress, we can write our FileUploader. Creating the request body for our upload request involves using a CountingRequestBody that reports progress and completion to a PublishSubject (or another reactive type).

private fun createUploadRequestBody(
file: File,
mimeType: String,
progressEmitter: PublishSubject<Double>
): RequestBody {
val fileRequestBody = file.asRequestBody(mimeType.toMediaType())
return CountingRequestBody(fileRequestBody) {
bytesWritten, contentLength ->
val progress = 1.0 * bytesWritten / contentLength
if (progress >= 1.0) {
progressEmitter.onComplete()
}else{
progressEmitter.onNext(progress)
}
}
}

The upload request consists of using the Retrofit function we implemented at the beginning, providing the file details and the created request body that will count progress. The Retrofit definition and the format of the request parts will depend on how each particular API is put together. Here we are using a request that contains various plaintext parts for the file details and then one for the file to be uploaded.

private fun createUploadRequest(
filename: String,
file: File,
mimeType: String,
progressEmitter: PublishSubject<Double>
): Single<AttachmentUploadedRemoteDto> {
val requestBody = createUploadRequestBody(file, mimeType, progressEmitter)
return remoteApi.attachFile(
filename = filename.toPlainTextBody(),
mimeType = mimeType.toPlainTextBody(),
fileSize = file.length().toString().toPlainTextBody(),
files = MultipartBody.Part.createFormData(
"files",
filename,
requestBody
)
)
}

Our main upload function can put together all of these parts to create a single result stream. We will be able to observe this to get progress updates as well as the final result.

fun uploadAttachment(
filename: String, file: File, mimeType: String
): Observable<AttachmentUploadRemoteResult> {
val progressEmitter = PublishSubject.create<Double>()
val uploadRequest = createUploadRequest(
filename, file, mimeType, progressEmitter
)
val uploadResult = uploadRequest
.map<AttachmentUploadRemoteResult> {
CountingRequestResult.Completed(it)
}
.toObservable()
val progressResult = progressEmitter
.map<AttachmentUploadRemoteResult> {
CountingRequestResult.Progress(it)
}
return progressResult.mergeWith(uploadResult)}
}
typealias AttachmentUploadRemoteResult = CountingRequestResult<AttachmentUploadedRemoteDto>

We can now upload a file to our API and update a view as the request progresses, which is nice for noticeably long operations like uploading larger files.

val bag = CompositeDisposable()
val uploader = AttachmentUploader()
uploader.uploadAttachment(file.name, file, "application/octet-stream")
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribeBy(
onError = { error ->
// Display error alert
},
onComplete = {
// Display completed Snackbar
},
onNext = { progress ->
// Update progress bar
}
)
.addTo(bag)

Wrap up

Monitoring the progress of a web request may not be immediately obvious when reading through the Retrofit API, however, the powerful APIs of OkHttp and Okio can get the job done nicely. The solution we have developed can be used for any web request, as the counting process can be wrapped around any RequestBody that needs to be sent in a request.

Thanks for reading and happy coding! 🙏

All source code : https://gist.github.com/VonLisboa/8a0fcafa07ccbf7c4337ef5ebcc27832