![Page 1: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/1.jpg)
Everything You Wanted to
Know
About Writing Async,
Concurrent HTTP Apps in Java
![Page 2: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/2.jpg)
Agenda
• Mostly this:
![Page 3: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/3.jpg)
Agenda
• And this:
![Page 4: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/4.jpg)
Agenda
• And this:
![Page 5: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/5.jpg)
About your speaker
linkd.in/jbaruch
stackoverflow.com/users/402053/jbaruch
![Page 6: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/6.jpg)
What Frog?
![Page 7: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/7.jpg)
What Frog?
![Page 8: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/8.jpg)
What Frog?
![Page 9: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/9.jpg)
What Frog?
![Page 10: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/10.jpg)
![Page 11: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/11.jpg)
![Page 12: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/12.jpg)
Requirements
– parallel file Downloads
– Parallel file parts
– interrupt/pause/resume
– Progress events
– Checksums caching
![Page 13: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/13.jpg)
First Association for “concurrent
downloader”
![Page 14: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/14.jpg)
![Page 15: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/15.jpg)
Lucky day: Download manager
written in Java!
![Page 16: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/16.jpg)
![Page 17: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/17.jpg)
Let’s look if we can use it!
1. No traceable license2. No website or docs3. No traceable sources4. It’s an app, not a lib
![Page 18: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/18.jpg)
![Page 19: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/19.jpg)
![Page 20: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/20.jpg)
Java.net.urlconnection
1. Memory wasteful (buffering)
2. Minimal API
3. Blocking streams
![Page 21: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/21.jpg)
![Page 22: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/22.jpg)
What we’re looking for
1. Async/non-blocking
2. Event callbacks
![Page 23: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/23.jpg)
What is IT going to take
1. Reactor
2. nio
![Page 24: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/24.jpg)
Welcome to the reactor
![Page 25: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/25.jpg)
– pattern for lightweight concurrency
– Event driven
– Threads reuse
– Uses non-blocking Io
![Page 26: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/26.jpg)
Original pattern
http://www.dre.vanderbilt.edu/~schmidt/PDF/reactor-siemens.pdf
![Page 27: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/27.jpg)
Guess the author by the
diagram
http://gee.cs.oswego.edu/dl/cpjslides/nio.pdf
![Page 28: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/28.jpg)
In Java, Reactor means NIO
![Page 29: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/29.jpg)
Selector as a multiplexer
![Page 30: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/30.jpg)
Java version - Registering
SocketChannel channel= SocketChannel.open();
socketChannel.connect(new
InetSocketAddress("http://remote.com", 80));
...
Selector selector = Selector.open();
channel.configureBlocking(false);
SelectionKey k = channel.register(selector,
SelectionKey.OP_READ);
k.attach(handler);
![Page 31: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/31.jpg)
Java version - Dispatcher
while (!Thread.interrupted()) {
selector.select();
Set selected = selector.selectedKeys();
Iterator it = selected.iterator();
while (it.hasNext())
SelectionKey k = (SelectionKey)(it.next();
((Runnable)(k.attachment())).run();
selected.clear();
}
![Page 32: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/32.jpg)
Handling reactor events is complex
– Need to maintain state
– Buffering – assembling chunks
– Coordinating async events
![Page 33: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/33.jpg)
![Page 34: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/34.jpg)
Nio libraries
– Most of them are servers–Netty, grizzly, etc.
– Apache Mina– Apache HTTP components asyncclient– Ning http client
![Page 35: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/35.jpg)
Nio libraries
– Most of them are servers–Netty, grizzly, etc.
– Apache Mina– Apache HTTP components asyncclient– Ning http client
![Page 36: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/36.jpg)
– Client and server nio library
– Evolved from netty
– Latest release October 2012
![Page 37: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/37.jpg)
![Page 38: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/38.jpg)
Nio libraries
– Most of them are servers–Netty, grizzly, etc
– Apache Mina– Apache HTTP components asyncclient– Ning http client
![Page 39: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/39.jpg)
![Page 40: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/40.jpg)
Ning’s async http client
![Page 41: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/41.jpg)
![Page 42: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/42.jpg)
![Page 43: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/43.jpg)
Here it is!
![Page 44: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/44.jpg)
try (AsyncHttpClient asyncHttpClient = new AsyncHttpClient()) {
ListenableFuture<Response> future = asyncHttpClient.prepareGet(
"http://oss.jfrog.org/api/system/ping").execute(
new AsyncCompletionHandler<Response>() {
@Override
public Response onCompleted(Response response) {
System.out.println(response.getResponseBody());
return response;
}
@Override
public void onThrowable(Throwable t) {
t.printStackTrace();
}
});
Response response = future.get();
}
![Page 45: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/45.jpg)
![Page 46: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/46.jpg)
HAC Concepts
– Request producer
– Response consumer
![Page 47: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/47.jpg)
try(CloseableHttpAsyncClientasyncHttpClient=HttpAsyncClients.createDefault()){asyncHttpClient.start();Future<HttpResponse>future=asyncHttpClient.execute(HttpAsyncMethods.createGet("http://oss.jfrog.org/api/system/ping"),newAsyncByteConsumer<HttpResponse>(){@OverrideprotectedvoidonResponseReceived(finalHttpResponseresponse){System.out.println(response.getStatusLine().getReasonPhrase());}@OverrideprotectedvoidonByteReceived(finalCharBufferbuf,finalIOControlioctrl){}@OverrideprotectedvoidreleaseResources(){}@OverrideprotectedHttpResponsebuildResult(finalHttpContextcontext){return(HttpResponse)context.getAttribute("http.response");}},null);HttpResponseresponse=future.get();}
![Page 48: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/48.jpg)
![Page 49: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/49.jpg)
Choosing between ning and http
asyncclient
![Page 50: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/50.jpg)
"All problems in computer science can
be solved by another level of
indirection"David
Wheeler
![Page 51: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/51.jpg)
public interface HttpProviderDownloadHandler {
void onResponseReceived(int statusCode, Map<String, List<String>> headers);
boolean onBytesReceived(ByteBuffer buf);
void onFailed(Throwable error);
void onCanceled();
void onCompleted();}
![Page 52: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/52.jpg)
Head to head
Feature/Library Ning client Http Async Client
Maturity Good Very new (early 2014)
Download cancelation Easy With bugs
Progress hooks Events not granular enough
Just use onByteReceived()
Documentation A bit sparse Minimal
Server-side counterpart None, client only org.apache.httpcomponentshttpcore-nio
![Page 53: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/53.jpg)
Performance?
0
100
200
300
400
500
600
700
800
900
Small file Medium file Large file
Ning
AHAC
http://blogs.atlassian.com/2013/07/http-client-performance-io/
![Page 54: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/54.jpg)
Rfc2616: a universe of its own
![Page 55: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/55.jpg)
![Page 56: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/56.jpg)
Confused?
![Page 57: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/57.jpg)
Just read some stackoverflow
(and improve your rep as you go)
![Page 58: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/58.jpg)
And that one for discovering that range header is lost on redirect
![Page 59: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/59.jpg)
Question!
What should be
content-length
when using
compression?
![Page 60: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/60.jpg)
![Page 61: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/61.jpg)
![Page 62: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/62.jpg)
https://github.com/http2/http2-spec/issues/46
![Page 63: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/63.jpg)
Question!
Why when redirected to CDN all the chunks start from zero?
![Page 64: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/64.jpg)
HttpAsyncClientBuilder builder = HttpAsyncClients.custom();// add redirect strategy that copies "range" headers, if existbuilder.setRedirectStrategy(new DefaultRedirectStrategy() {
@Overridepublic HttpUriRequest getRedirect(HttpRequest request, HttpResponse response,
HttpContext context)HttpUriRequest redirectRequest = super.getRedirect(request, response, context);// copy "Range" headers, if existHeader[] rangeHeaders = request.getHeaders(HttpHeaders.RANGE);if (rangeHeaders != null) {
for (Header header : rangeHeaders) {redirectRequest.addHeader(header);
}}return redirectRequest;
}});
![Page 65: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/65.jpg)
Question!
How many simultaneous
connections should I open?
![Page 66: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/66.jpg)
![Page 67: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/67.jpg)
![Page 68: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/68.jpg)
![Page 69: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/69.jpg)
![Page 70: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/70.jpg)
Question!
What’s wrong
with the
following
code?
![Page 71: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/71.jpg)
public static String encodeUrl(String urlStr) {URLEncoder.encode(urlStr, "UTF-8");...
}
![Page 72: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/72.jpg)
Decoded URLs cannot be
re-encoded to the same form
http://example.com/?query=a&b==c
Cannot be decoded back after it was encoded:
http://example.com/?query=a%26b==c
![Page 73: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/73.jpg)
Don’t use java.net.URLEncoder
“Utility class for HTML form encoding. This class contains static methods for converting a String to the application/x-www-form-urlencoded MIME format.
For more information about HTML form encoding, consult the HTML specification.”
![Page 74: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/74.jpg)
AHC Alternatives
![Page 75: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/75.jpg)
![Page 76: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/76.jpg)
Question!
How do I
close a
socket
correctly?
![Page 77: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/77.jpg)
How hard can it be to close a
socket?
![Page 78: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/78.jpg)
The art of socket closing
http://www.safaribooksonline.com/library/view/http-the-definitive/1565925092/ch04s07.html
![Page 79: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/79.jpg)
Half-closed: no new customers
![Page 80: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/80.jpg)
Never block in socket close()
• The other side expects you
to clean up nicely
• It will give up on time out
• You will wait (forever)
![Page 81: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/81.jpg)
![Page 82: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/82.jpg)
Remember?
![Page 83: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/83.jpg)
Question!
How can I write
file parts
concurrently?
![Page 84: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/84.jpg)
– Write to separate files, combine on finish
– Write to same file, seeking to the right position
![Page 85: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/85.jpg)
![Page 86: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/86.jpg)
![Page 87: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/87.jpg)
Use FileChannel
• Implements SeekableByteChannel
java.nio.channels.FileChannel#write(java.nio.ByteBuffer src, long position)
![Page 88: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/88.jpg)
Download progress tracking
• PersistentFileProgressInfo– Save the total size, sha1, number of parts
– State of each part (offset, size, completed...)
FileProgressInfo FilePartProgressInfo*
![Page 89: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/89.jpg)
File Locking
![Page 90: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/90.jpg)
File locking Levels
– VM level
– OS level
![Page 91: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/91.jpg)
OS level File locking
• Multiple downloader instances writing to
the same file
• Needed for writing:
– Partial download file
– Persistent download progress
![Page 92: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/92.jpg)
FileLock lock = fileChannel.tryLock();//Non-shared: (0L, Long.MAX_VALUE, false)
if (lock == null) {throw new OverlappingFileLockException();
}return lock;
}
OS Level File Locking -
Exclusive
![Page 93: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/93.jpg)
private FileLock lock(FileChannel fileChannel) throws IOException {FileLock lock = fileChannel.tryLock(Long.MAX_VALUE - 1, 1, false);if (lock == null) {
throw new OverlappingFileLockException();}return lock;
}
OS Level File Locking –
Advisory exclusive
WTF?!
![Page 94: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/94.jpg)
VM Level File Locking
![Page 95: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/95.jpg)
VM Level File Locking– Prevent same VM threads writing to the file when we
started closing it– Closing sequence:
– Release file locks– Close channels– Rename a file to it's final name (remove .part)– Erase progress info
![Page 96: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/96.jpg)
VM Level File LockingReentrantReadWriteLock.ReadLock writeToFileLock = rwl.readLock();ReentrantReadWriteLock.WriteLock closeFileLock = rwl.writeLock();
public void close() throws IOException {this.closeFileLock.lock();
}
public int write(int partIndex, ByteBuffer buf) {if (!this.writeToFileLock.tryLock()) {
throw new IllegalStateException("File is being closed");}...
}
![Page 97: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/97.jpg)
What’s next?
![Page 98: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/98.jpg)
http/2– Mostly standardizing Google's spdy– Header compression– multiplexing– Prioritization– Server push– On the way clear some stuff– E.g. compressed content length
![Page 99: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/99.jpg)
Ease the load
![Page 100: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/100.jpg)
Links!
• RTFM: RFC 2616
• Ultimate book: HTTP: The Definitive
Guide
– Amazon
– Safari
• Reactor pattern
• Doug Lea on NIO
![Page 101: Presentation: Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask](https://reader034.vdocuments.site/reader034/viewer/2022042504/55a8676b1a28abff3f8b484e/html5/thumbnails/101.jpg)
No, Thank you!