<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Mathias's blog]]></title><description><![CDATA[Mathias's blog]]></description><link>https://mathias-vandaele.dev</link><generator>RSS for Node</generator><lastBuildDate>Sat, 06 Jun 2026 16:24:21 GMT</lastBuildDate><atom:link href="https://mathias-vandaele.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Why I Stopped Calling OSRM Over HTTP (and Wrote Rust Bindings Instead)]]></title><description><![CDATA[Some problems aren't about making the HTTP API faster. They're about the HTTP API being the wrong tool entirely.
I needed to generate a dataset of 500 million (origin, destination, travel_time) triple]]></description><link>https://mathias-vandaele.dev/why-i-stopped-calling-osrm-over-http-and-wrote-rust-bindings-instead</link><guid isPermaLink="true">https://mathias-vandaele.dev/why-i-stopped-calling-osrm-over-http-and-wrote-rust-bindings-instead</guid><dc:creator><![CDATA[Mathias Vandaele]]></dc:creator><pubDate>Thu, 16 Apr 2026 08:35:47 GMT</pubDate><content:encoded><![CDATA[<p>Some problems aren't about making the HTTP API faster. They're about the HTTP API being the wrong tool entirely.</p>
<p>I needed to generate a dataset of <strong>500 million (origin, destination, travel_time) triplets</strong> over French address data to train a travel time embedding model, encoding ~25 million addresses from the <a href="https://adresse.data.gouv.fr/">Base Adresse Nationale</a> into 64-dimensional vectors where distances approximate real-world road travel times.</p>
<p>At 500M rows, even an optimistic 1ms per HTTP request (localhost, keep-alive, no JSON parsing overhead) puts you at <strong>~139 hours of wall time</strong>. That's not a latency problem. That's a fundamental mismatch between the tool and the task. The only path forward was to call OSRM in-process and eliminate the transport layer entirely.</p>
<p>So I wrote <a href="https://crates.io/crates/osrm-binding"><code>osrm-binding</code></a>: idiomatic Rust bindings that call OSRM's C++ engine <strong>in-process</strong>, skipping the network entirely.</p>
<h2>The Problem with the HTTP API</h2>
<p>OSRM ships with a wonderful HTTP server. For most use cases — a web app displaying a route, a backend computing occasional ETAs — it's perfect. But HTTP has costs that compound quickly:</p>
<ul>
<li><p><strong>Serialization/deserialization</strong> of JSON on every request</p>
</li>
<li><p><strong>TCP overhead</strong> even on localhost</p>
</li>
<li><p><strong>Latency floor</strong> that's hard to get below ~1ms even with keep-alive</p>
</li>
</ul>
<p>When you need to query 10,000 routes in a tight loop (say, to generate training data or compute a distance matrix for route optimization), you're burning time on the transport layer, not the routing itself.</p>
<p>The alternative is to link against OSRM's C++ library directly and call it in-process. OSRM exposes a C++ API for exactly this. The catch: you need to bridge that API to your language of choice. <code>osrm-binding</code> does that bridge for Rust.</p>
<h2>What the Crate Provides</h2>
<pre><code class="language-toml"># Cargo.toml
osrm-binding = "0.1.7"
</code></pre>
<p>The crate exposes four main capabilities, all through a single <code>OsrmEngine</code> handle:</p>
<h3>Engine Initialization</h3>
<pre><code class="language-rust">use osrm_binding::{OsrmEngine, Algorithm};

let engine = OsrmEngine::new("/path/to/france-latest.osrm", Algorithm::MLD)
    .expect("Failed to initialize OSRM engine");
</code></pre>
<p>You choose between <strong>MLD</strong> (Multi-Level Dijkstra — better for dynamic scenarios like traffic) and <strong>CH</strong> (Contraction Hierarchies — faster for static graphs). Both are supported.</p>
<h3>Route Calculation</h3>
<pre><code class="language-rust">use osrm_binding::{RouteRequestBuilder, Point};

let request = RouteRequestBuilder::default()
    .points(vec![
        Point { longitude: 2.3522, latitude: 48.8566 }, // Paris
        Point { longitude: 5.3698, latitude: 43.2965 }, // Marseille
    ])
    .build()
    .unwrap();

let result = engine.route(&amp;request).unwrap();
println!("{:?}", result.routes.first().unwrap());
</code></pre>
<h3>Distance/Duration Table (the workhorse for matrix problems)</h3>
<p>This is the call I use most heavily. Given N sources and M destinations, it returns an N×M matrix of durations and distances in one shot:</p>
<pre><code class="language-rust">use osrm_binding::{TableRequest, Point};

let request = TableRequest {
    sources: vec![
        Point { longitude: 2.3522, latitude: 48.8566 },
    ],
    destinations: vec![
        Point { longitude: 5.3698, latitude: 43.2965 }, // Marseille
        Point { longitude: 4.8357, latitude: 45.7640 }, // Lyon
    ],
};

let response = engine.table(&amp;request).unwrap();
println!("{:?}", response.durations);
</code></pre>
<h3>Simple Route (convenience wrapper)</h3>
<p>For when you just need duration and distance between two points without building a full request:</p>
<pre><code class="language-rust">use osrm_binding::Point;

let result = engine.simple_route(
    Point { longitude: 2.3522, latitude: 48.8566 },
    Point { longitude: 5.3698, latitude: 43.2965 },
).unwrap();

println!("Duration: {}s, Distance: {}m", result.duration, result.distance);
</code></pre>
<h3>Trip (TSP Solver)</h3>
<p>OSRM also ships a trip endpoint that solves approximate TSP. Useful for delivery route optimization:</p>
<pre><code class="language-rust">use osrm_binding::{TripRequest, Point};

let request = TripRequest {
    points: vec![
        Point { longitude: 2.3522, latitude: 48.8566 },
        Point { longitude: 4.8357, latitude: 45.7640 },
        Point { longitude: 5.3698, latitude: 43.2965 },
    ],
};

let trip = engine.trip(&amp;request).unwrap();
println!("{:?}", trip);
</code></pre>
<h2>Performance: The Reason This Exists</h2>
<p>Here are benchmark numbers from <code>cargo bench</code>, computing batches of routes around Paris:</p>
<table>
<thead>
<tr>
<th>Scenario</th>
<th>Algorithm</th>
<th>Time</th>
</tr>
</thead>
<tbody><tr>
<td>10km radius, multiple routes</td>
<td>MLD</td>
<td>~5.5 ms</td>
</tr>
<tr>
<td>100km radius, multiple routes</td>
<td>MLD</td>
<td>~13.9 ms</td>
</tr>
<tr>
<td>10km radius, multiple routes</td>
<td>CH</td>
<td>~3.9 ms</td>
</tr>
<tr>
<td>100km radius, multiple routes</td>
<td>CH</td>
<td>~6.2 ms</td>
</tr>
</tbody></table>
<p>These are in-process calls with no serialization, no TCP, no JSON parsing. Compare this to a typical localhost HTTP round-trip which rarely goes below 1–2ms <em>per single request</em> — and that's before you factor in JSON decode time on the response.</p>
<p>For my use case — generating 500M training triplets for a metric learning model over French address pairs — this difference isn't about speed, it's about feasibility. At 1ms per HTTP request, 500M rows would take ~139 hours. In-process, with the table API batching multiple destinations per call, the same dataset becomes tractable in a fraction of that time.</p>
<hr />
<h2>Setup: How the Build Works</h2>
<p>Here's something worth understanding: you don't install OSRM. <code>cargo build</code> handles everything.</p>
<p>The <a href="http://build.rs"><code>build.rs</code></a> script downloads the OSRM v6.0.0 source tarball directly from GitHub, decompresses it into the Cargo <code>OUT_DIR</code>, and builds it via CMake — all automatically, the first time you build your project. The OSRM C++ libraries end up statically linked into your binary. There's no separate installation step, no system-wide OSRM install required.</p>
<p>What you <em>do</em> need are the system libraries that OSRM itself depends on: Boost, TBB, libfmt, libbz2, and a few others. These are standard packages available in any Ubuntu/Debian repo.</p>
<h3>Ubuntu 24.04</h3>
<pre><code class="language-bash">sudo apt update
sudo apt install build-essential git cmake pkg-config \
                libbz2-dev libxml2-dev libzip-dev libboost-all-dev \
                lua5.2 liblua5.2-dev libtbb-dev libfmt-dev
</code></pre>
<p>Then just <code>cargo build --release</code>. The first build takes a while (it's compiling OSRM from source), but subsequent builds are cached.</p>
<h3>Docker</h3>
<p>The multi-stage Dockerfile installs build deps, lets <code>cargo build</code> handle the OSRM compilation, then copies the binary into a slim runtime image with only the shared <code>.so</code> files needed at runtime:</p>
<pre><code class="language-dockerfile">FROM rust:1.88.0-bookworm AS builder

WORKDIR /usr/src/app
COPY Cargo.toml Cargo.lock ./
COPY ./src ./src

RUN apt-get update &amp;&amp; apt-get install -y --no-install-recommends \
    cmake g++ gcc git \
    libboost1.81-all-dev libbz2-dev liblua5.4-dev \
    libtbb-dev libxml2-dev libzip-dev lua5.4 \
    make pkg-config libfmt-dev

RUN cargo build --release

FROM debian:bookworm-slim
WORKDIR /usr/src/app
COPY --from=builder /usr/src/app/target/release/my-bin ./

RUN apt-get update &amp;&amp; apt-get install -y --no-install-recommends \
    expat libboost-date-time1.81.0 libboost-iostreams1.81.0 \
    libboost-program-options1.81.0 libboost-thread1.81.0 \
    liblua5.4-0 libtbb12 &amp;&amp; \
    rm -rf /var/lib/apt/lists/* &amp;&amp; ldconfig /usr/local/lib

CMD ["./my-bin"]
</code></pre>
<p>Replace <code>my-bin</code> with your binary name. The builder needs the full <code>-dev</code> packages (headers + static libs for compilation), while the runtime image only needs the dynamic <code>.so</code> files.</p>
<h2>Preparing Your OSRM Data File</h2>
<p>If you haven't preprocessed an OSRM file before, the process is:</p>
<pre><code class="language-shell"># Download an extract (example: France from Geofabrik)
wget https://download.geofabrik.de/europe/france-latest.osm.pbf

# Extract with a profile (car, bicycle, foot)
osrm-extract -p /usr/share/osrm/profiles/car.lua france-latest.osm.pbf

# Contract (for CH algorithm) or partition+customize (for MLD)
osrm-contract france-latest.osrm
# OR for MLD:
# osrm-partition france-latest.osrm
# osrm-customize france-latest.osrm
</code></pre>
<p>The resulting <code>.osrm</code> file (along with its companion files in the same directory) is what you pass to <code>OsrmEngine::new</code>.</p>
<hr />
<h2>When to Use This vs. the HTTP Client</h2>
<table>
<thead>
<tr>
<th></th>
<th><code>osrm-binding</code></th>
<th>HTTP client</th>
</tr>
</thead>
<tbody><tr>
<td>Latency per query</td>
<td>~0.1ms (in-process)</td>
<td>~1–5ms (localhost)</td>
</tr>
<tr>
<td>Throughput</td>
<td>Very high</td>
<td>Moderate</td>
</tr>
<tr>
<td>Setup complexity</td>
<td><code>cargo build</code> fetches &amp; compiles OSRM automatically; system libs (Boost, TBB, fmt) needed</td>
<td>Just needs a running server</td>
</tr>
<tr>
<td>Deployment</td>
<td>Binary links OSRM statically/dynamically</td>
<td>Separate OSRM process</td>
</tr>
<tr>
<td>Best for</td>
<td>Batch processing, training data, matrix ops</td>
<td>Web backends, occasional queries</td>
</tr>
</tbody></table>
<p>If you're running OSRM on a separate machine or want to keep routing as an independent service, use the HTTP API. If you're doing heavy batch computation and OSRM can live on the same machine as your process, the binding pays off immediately.</p>
<hr />
<h2>What's Next</h2>
<p>The crate is at v0.1.7 and covers the main OSRM endpoints I needed: <code>route</code>, <code>table</code>, <code>trip</code>, and <code>simple_route</code>. OSRM also exposes <code>nearest</code> and <code>match</code> (map matching) endpoints which aren't bound yet — contributions welcome.</p>
<p>The repo is at <a href="https://github.com/mathias-vandaele/osrm-binding">github.com/mathias-vandaele/osrm-binding</a>. It's MIT licensed, has integration tests (set <code>OSRM_TEST_DATA_PATH</code> to your <code>.osrm</code> file), and benchmarks you can run with <code>cargo bench</code>.</p>
<p>If you're building anything in the routing/geospatial space in Rust and hitting performance ceilings with HTTP, give it a try.</p>
]]></content:encoded></item><item><title><![CDATA[Static dispatch vs dynamic dispatch in Rust, how to dramatically improve performances + Java 21 bonus]]></title><description><![CDATA[Before diving deeper, let's understand the key differences between static and dynamic dispatching.
When you have a trait (interface for those unfamiliar with rust), you have to put the function signature of what all classes or struct will implement.
...]]></description><link>https://mathias-vandaele.dev/static-dispatch-vs-dynamic-dispatch-in-rust-how-to-dramatically-improve-performances-java-21-bonus</link><guid isPermaLink="true">https://mathias-vandaele.dev/static-dispatch-vs-dynamic-dispatch-in-rust-how-to-dramatically-improve-performances-java-21-bonus</guid><category><![CDATA[static dispatch]]></category><category><![CDATA[dynamic dispatch]]></category><category><![CDATA[Rust]]></category><category><![CDATA[Java]]></category><category><![CDATA[Java21]]></category><dc:creator><![CDATA[Mathias Vandaele]]></dc:creator><pubDate>Wed, 28 Feb 2024 20:08:21 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1709151286035/127e8313-fed4-4ab6-ac92-62fbd04d2f0a.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Before diving deeper, let's understand the key differences between static and dynamic dispatching.</p>
<p>When you have a trait (interface for those unfamiliar with rust), you have to put the function signature of what all classes or struct will implement.</p>
<p>Let's call this trait <code>Vehicle</code> that could declare methods such as <code>drive</code> <code>get_hp</code> and many others. There could be many classes implementing this interface.</p>
<p>Now, typically in many languages (like Java), when the program meets an instance of <code>Vehicle</code>, It will dynamically find which implementation it has to execute, like doing an <code>instanceof</code> and executing the right implementation (the JVM indeed performs a type check, but not an <code>instanceof</code>)</p>
<p>Dynamic dispatching is this process of finding the proper implementation of a polymorphic operation.</p>
<p>While traditional approaches exist, this session dives into optimized techniques that can unlock up to 10x performance gains! We'll explore how these strategies leverage the new features introduced in Java 21.</p>
<h3 id="heading-the-code">The code</h3>
<p>First let's implement a simple trait, here <code>Vehicle</code></p>
<pre><code class="lang-rust"><span class="hljs-class"><span class="hljs-keyword">trait</span> <span class="hljs-title">Vehicle</span></span> {
    <span class="hljs-function"><span class="hljs-keyword">fn</span> <span class="hljs-title">get_hp</span></span>(&amp;<span class="hljs-keyword">self</span>) -&gt; <span class="hljs-built_in">u32</span>;
}
</code></pre>
<p>Then we will create two structs and implement them with this trait.</p>
<pre><code class="lang-rust"><span class="hljs-class"><span class="hljs-keyword">struct</span> <span class="hljs-title">Mercedes</span></span> {
    hp: <span class="hljs-built_in">u32</span>,
}

<span class="hljs-class"><span class="hljs-keyword">struct</span> <span class="hljs-title">AMG</span></span> {
    hp: <span class="hljs-built_in">u32</span>,
}

<span class="hljs-keyword">impl</span> Vehicle <span class="hljs-keyword">for</span> Mercedes {
    <span class="hljs-function"><span class="hljs-keyword">fn</span> <span class="hljs-title">get_hp</span></span>(&amp;<span class="hljs-keyword">self</span>) -&gt; <span class="hljs-built_in">u32</span> {
        <span class="hljs-keyword">self</span>.hp
    }
}

<span class="hljs-keyword">impl</span> Vehicle <span class="hljs-keyword">for</span> AMG {
    <span class="hljs-function"><span class="hljs-keyword">fn</span> <span class="hljs-title">get_hp</span></span>(&amp;<span class="hljs-keyword">self</span>) -&gt; <span class="hljs-built_in">u32</span> {
        <span class="hljs-keyword">self</span>.hp * <span class="hljs-number">2</span>
    }
}
</code></pre>
<p>Now, let's write a function that takes 2 instances of <code>Vehicle</code> and returns one of them. Like in this example:</p>
<pre><code class="lang-rust"><span class="hljs-function"><span class="hljs-keyword">fn</span> <span class="hljs-title">test_dynamic_dispatching</span></span>&lt;<span class="hljs-symbol">'a</span>&gt;(x: &amp;<span class="hljs-symbol">'a</span> AMG, y: &amp;<span class="hljs-symbol">'a</span> Mercedes) -&gt; &amp;<span class="hljs-symbol">'a</span> <span class="hljs-keyword">dyn</span> Vehicle {
    <span class="hljs-keyword">if</span> x.get_hp() &gt; y.get_hp() {
        <span class="hljs-keyword">return</span> x;
    }
    y
}
</code></pre>
<p>The compiler does not have prior knowledge of which concrete instance of <code>Vehicle</code> will be returned during runtime. This uncertainty necessitates the use of the <code>dyn</code> keyword. Unlike generic parameters or <code>impl Trait</code>, where the compiler can infer the concrete type being passed, dynamic dispatching requires explicit indication using <code>dyn</code>.</p>
<p>So calling <code>get_hp</code> on the returned instance of <code>Vehicle</code> will be done using a vtable, containing all the possible implementations and execute the right one.</p>
<h3 id="heading-understanding-static-dispatching">Understanding static dispatching.</h3>
<p>Now we are going to do something new in our code, we are going to add an enum, that contains structs that implements our trait.</p>
<pre><code class="lang-rust"><span class="hljs-class"><span class="hljs-keyword">enum</span> <span class="hljs-title">VehicleDispatcher</span></span>&lt;<span class="hljs-symbol">'a</span>&gt; {
    AMG(&amp;<span class="hljs-symbol">'a</span> AMG),
    Mercedes(&amp;<span class="hljs-symbol">'a</span> Mercedes),
}
</code></pre>
<p>And we will make the same function as we declared above, but slightly different:</p>
<pre><code class="lang-rust"><span class="hljs-function"><span class="hljs-keyword">fn</span> <span class="hljs-title">test_static_dispatching</span></span>&lt;<span class="hljs-symbol">'a</span>&gt;(x: &amp;<span class="hljs-symbol">'a</span> AMG, y: &amp;<span class="hljs-symbol">'a</span> Mercedes) -&gt; VehicleDispatcher&lt;<span class="hljs-symbol">'a</span>&gt; {
    <span class="hljs-keyword">if</span> x.get_hp() &gt; y.get_hp() {
        <span class="hljs-keyword">return</span> VehicleDispatcher::AMG(&amp;x);
    }
    VehicleDispatcher::Mercedes(&amp;y)
}
</code></pre>
<p>Now, when calling this function, we will have to do a match on it, in order to call the function on it. This is now static dispatching, there is no more performance loss looking at a map in order to find the proper implementation.</p>
<h3 id="heading-performance-test">Performance test</h3>
<p>Using <code>black_box</code> is critical to prevent the compiler from "optimizing" to much and remove this part of the code.</p>
<p>Do not forget to run or build this code using <code>--release</code></p>
<pre><code class="lang-rust">    <span class="hljs-keyword">let</span> m = Mercedes { hp: <span class="hljs-number">234</span> };
    <span class="hljs-keyword">let</span> a = AMG { hp: <span class="hljs-number">987</span> };

    <span class="hljs-keyword">let</span> start1 = Instant::now();
    <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> <span class="hljs-number">0</span>..<span class="hljs-number">100000000</span> {
        <span class="hljs-keyword">let</span> v = test_dynamic_dispatching(&amp;a, &amp;m);
        black_box(v.get_hp());
    }
    <span class="hljs-keyword">let</span> duration1 = start1.elapsed();
    <span class="hljs-built_in">println!</span>(<span class="hljs-string">"Time elapsed in dynamic dispatching is: {:?}"</span>, duration1);


    <span class="hljs-keyword">let</span> start2 = Instant::now();
    <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> <span class="hljs-number">0</span>..<span class="hljs-number">100000000</span> {
        <span class="hljs-keyword">let</span> v = test_static_dispatching(&amp;a, &amp;m);
        <span class="hljs-keyword">match</span> v {
            VehicleDispatcher::AMG(amg) =&gt; { black_box(amg.get_hp()); }
            VehicleDispatcher::Mercedes(mer) =&gt; { black_box(mer.get_hp()); }
        };
    }
    <span class="hljs-keyword">let</span> duration2 = start2.elapsed();
    <span class="hljs-built_in">println!</span>(<span class="hljs-string">"Time elapsed in static dispatching is: {:?}"</span>, duration2);
</code></pre>
<p>While running the code, we get this gain in performance.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1709149434736/4420fa22-c402-408e-91a0-6d7925af8cc3.png" alt class="image--center mx-auto" /></p>
<p>Amazing! Isn't it?</p>
<h3 id="heading-conclusion">Conclusion</h3>
<p>For such a slight difference in the code, we get such an improvement that can be life-changing when performance is critical.</p>
<p>That is the reason we have crates that created macros to simplify the process of doing so, such as <code>enum_dispatch</code> <a target="_blank" href="https://docs.rs/enum_dispatch/latest/enum_dispatch/">https://docs.rs/enum_dispatch/latest/enum_dispatch/</a></p>
<p>I said I would write about Java 21, but do you see a similarity?</p>
<p>I saw many people around me not understanding the sealed type's benefits and being baffled by the <code>permits</code> keyword. This video from a Youtuber I like made me laugh: <a target="_blank" href="https://youtu.be/w87od6DjzAg?si=KxGea4GhmzsG9oeb&amp;t=480">https://youtu.be/w87od6DjzAg?si=KxGea4GhmzsG9oeb&amp;t=480</a></p>
<p>Does everything click now?</p>
<p>Java knows all possible subclasses/implementations because you explicitly list them in the permits clause. The compiler will ensure these are the only extensions/implementations allowed. This means the type system can be fully checked for completeness at compile time.</p>
<p>This is static dispatching introduced to Java, and that is amazing. Behind the door, it is precisely what we have done before by listing all the subclasses/implementations in an enum, but without our knowledge.</p>
<p>I hope you liked this article, and, as always, you can find the code on my GitHub.</p>
<p><a target="_blank" href="https://github.com/mathias-vandaele/dynamic-vs-static-dispatching">https://github.com/mathias-vandaele/dynamic-vs-static-dispatching</a></p>
]]></content:encoded></item><item><title><![CDATA[An interesting way of solving this google foo bar challenge.]]></title><description><![CDATA[I felt privileged to receive an invitation for the Google Foo Bar Challenge, a discreet hiring process by Google (https://www.turing.com/kb/foobar-google-secret-hiring-technique).
Although I've nearly completed all the exercises, there's one in parti...]]></description><link>https://mathias-vandaele.dev/an-interesting-way-of-solving-this-google-foo-bar-challenge</link><guid isPermaLink="true">https://mathias-vandaele.dev/an-interesting-way-of-solving-this-google-foo-bar-challenge</guid><category><![CDATA[google foobar]]></category><category><![CDATA[Java]]></category><category><![CDATA[leetcode]]></category><category><![CDATA[leetcode-solution]]></category><dc:creator><![CDATA[Mathias Vandaele]]></dc:creator><pubDate>Tue, 23 Jan 2024 19:32:18 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1706038284895/fd0da30c-ea23-47c4-8aeb-fa4c1dedd562.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>I felt privileged to receive an invitation for the Google Foo Bar Challenge, a discreet hiring process by Google (<a target="_blank" href="https://www.turing.com/kb/foobar-google-secret-hiring-technique">https://www.turing.com/kb/foobar-google-secret-hiring-technique</a>).</p>
<p><a target="_blank" href="https://www.turing.com/kb/foobar-google-secret-hiring-technique">Although I've nearly completed all the exercises, there's one</a> in particular that I found quite intriguing.</p>
<p>You can access it here: <a target="_blank" href="https://github.com/mathias-vandaele/google-foobar-challenge/blob/master/src/main/java/dev/mathias/exercise_3_2/Solution.java">https://github.com/mathias-vandaele/google-foobar-challenge/blob/master/src/main/java/dev/mathias/exercise_3_2/Solution.java</a></p>
<h1 id="heading-prepare-the-bunnies-escapehttpsgithubcommathias-vandaelegoogle-foobar-challengeblobmastersrcmainjavadevmathiasexercise32solutionjava"><a target="_blank" href="https://github.com/mathias-vandaele/google-foobar-challenge/blob/master/src/main/java/dev/mathias/exercise_3_2/Solution.java">Prepare the Bunnies' Escape</a></h1>
<p><a target="_blank" href="https://www.turing.com/kb/foobar-google-secret-hiring-technique"><code>You're awfully close to destroying the LAMBCHOP doomsday devic</code></a><a target="_blank" href="https://github.com/mathias-vandaele/google-foobar-challenge/blob/master/src/main/java/dev/mathias/exercise_3_2/Solution.java"><code>e and freeing Commander Lambda</code></a><code>'s bunny prisoners, but once they're free of the prison</code> <a target="_blank" href="https://github.com/mathias-vandaele/google-foobar-challenge/blob/master/src/main/java/dev/mathias/exercise_3_2/Solution.java"><code>blocks, the bunnies are going to need to escape Lambda's space station via the escape pods as quickly as possible. Unfortunately, the halls of the space station are a maze of corridors and dead ends that will be a deathtrap for the</code></a> <code>escaping bunnies. Fortunately, Commander Lambda has put you in charge of a remodeling project that will give you the opportunity to make things a little easier for the bunnies. Unfortunately (again), you can't just remove all obstacles between the bunnies and the escape pods - at most you can remove one wall per escape pod path, both to maintain structural integrity of the station and to avoid arousing Commander Lambda's suspicions.</code></p>
<p><code>You have maps of parts of the space station, each starting at a prison exit and ending at the door to an escape pod. The map is represented as a matrix of 0s and 1s, where 0s are passable space and 1s are impassable walls. The door out of the prison is at the top left (0,0) and the door into an escape pod is at the bottom right (w-1,h-1).</code></p>
<p><code>Write a function answer(map) that generates the length of the shortest path from the prison door to the escape pod, where you are allowed to remove one wall as part of your remodeling plans. The path length is the total number of nodes you pass through, counting both the entrance and exit nodes. The starting and ending positions are always passable (0). The map will always be solvable, though you may or may not need to remove a wall. The height and width of the map can be from 2 to 20. Moves can only be made in cardinal directions; no diagonal moves are allowed.</code></p>
<h1 id="heading-the-solution">The solution</h1>
<p>Many online solutions employ a dual Dijkstra's algorithm and utilize a double for loop to discover the shortest paths both from the starting point to the endpoint and vice versa. The objective is to identify the optimal location for breaking a wall, allowing the determination of the overall shortest path.</p>
<p>I believe this solution is not only inefficient, but can not be generalized, what if we are allowed to break more than one wall this algorithm becomes useless. This solution also has an early return, meaning the best solution can be found really really fast.</p>
<p>My chosen approach diverges significantly; I intend to transform the maze into a heat map. As the bunny takes more steps, the corresponding points on the map become hotter.</p>
<p>I subsequently determine the threshold for considering a non wall-crossing path as worthwhile. I arbitrarily set this threshold at 100,000 steps. This implies that I find it acceptable to explore a path that avoids wall crossings but is still at least 100,000 steps behind a path that did involve wall crossings.</p>
<p>I will make it into steps to give an example:</p>
<p>Here is the maze:</p>
<pre><code class="lang-java">{
        {<span class="hljs-number">0</span>, <span class="hljs-number">1</span>, <span class="hljs-number">1</span>, <span class="hljs-number">0</span>},
        {<span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">1</span>},
        {<span class="hljs-number">1</span>, <span class="hljs-number">1</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>},
        {<span class="hljs-number">1</span>, <span class="hljs-number">1</span>, <span class="hljs-number">1</span>, <span class="hljs-number">0</span>}
}
</code></pre>
<p>Here is the heating map associated (Note that I am using <code>MAX_INTEGER</code> to represent an unexplored node):</p>
<pre><code class="lang-json">[[<span class="hljs-number">0</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>],
[<span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>],
[<span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>],
[<span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>]]
</code></pre>
<p>Few steps after, we are ending up with this:</p>
<pre><code class="lang-json">[<span class="hljs-number">0</span>, <span class="hljs-number">100000</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>]
[<span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>]
[<span class="hljs-number">100001</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>]
[<span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>]
</code></pre>
<p>When a wall is destroyed, I increase the value in the heatmap for the preceding cell by 100,000 and update the current cell accordingly. In the absence of a wall, I simply increment the value by one. This approach enables the re-exploration of a path as long as a better score is achieved.</p>
<pre><code class="lang-json">[<span class="hljs-number">0</span>, <span class="hljs-number">100002</span>, <span class="hljs-number">100003</span>, <span class="hljs-number">100004</span>]
[<span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">3</span>, <span class="hljs-number">100005</span>]
[<span class="hljs-number">100001</span>, <span class="hljs-number">100004</span>, <span class="hljs-number">4</span>, <span class="hljs-number">5</span>]
[<span class="hljs-number">2147483647</span>, <span class="hljs-number">2147483647</span>, <span class="hljs-number">100004</span>, <span class="hljs-number">6</span>]
</code></pre>
<p>And when we finally found the end of the maze, we return early with the number of steps we have achieved.</p>
<h1 id="heading-conclusion">Conclusion</h1>
<p>I believe that employing a combination of a heatmap and Dijkstra's algorithm enhances the effectiveness of resolving this LeetCode problem compared to the conventional technique. This approach boasts a simpler time complexity and is capable of determining the shortest paths even when breaking more than one wall, possibly up to two or three walls.</p>
]]></content:encoded></item><item><title><![CDATA[Implementing onion architecture using Rust]]></title><description><![CDATA[Today, I will implement onion architecture using rust, we will make a simple with Actix, which is a compelling web framework. (https://actix.rs/) You will find here how I have implemented it: https://github.com/mathias-vandaele/receipts_inserter_onio...]]></description><link>https://mathias-vandaele.dev/implementing-onion-architecture-using-rust</link><guid isPermaLink="true">https://mathias-vandaele.dev/implementing-onion-architecture-using-rust</guid><category><![CDATA[clean code]]></category><category><![CDATA[Clean Architecture]]></category><category><![CDATA[Rust]]></category><category><![CDATA[actix]]></category><dc:creator><![CDATA[Mathias Vandaele]]></dc:creator><pubDate>Sun, 20 Aug 2023 12:46:48 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1692535640935/fa2b03cf-7ac5-4c16-8f5c-0894724065d9.avif" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Today, I will implement onion architecture using rust, we will make a simple with Actix, which is a compelling web framework. (<a target="_blank" href="https://actix.rs/">https://actix.rs/</a>) You will find here how I have implemented it: <a target="_blank" href="https://github.com/mathias-vandaele/receipts_inserter_onion">https://github.com/mathias-vandaele/receipts_inserter_onion</a> First, let's try to understand what is the onion architecture</p>
<h2 id="heading-the-onion-architecture">The onion architecture</h2>
<p>Onion architecture is a way to architecture your project in a layered manner.</p>
<p>This technic aims to create better separation of concerns, testability, maintainability, etc. It respects every SOLID principle. Here is a corresponding image of how onion architecture is layered.</p>
<p><img src="https://o.quizlet.com/lYFg3w5xFTOHiwautbnHTQ.png" alt="Clean Architecture Flashcards | Quizlet" class="image--center mx-auto" /></p>
<p>The way I made it is a little bit different, but I find it more straightforward and more understandable</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1692525029053/4943fe74-a887-4914-915b-c79725b83dcd.png" alt class="image--center mx-auto" /></p>
<ul>
<li>Domain:</li>
</ul>
<p>It's in charge of the Data Access Object (DAO), which is a fancy way of saying it handles how we talk to our data. The domain layer also has this thing called a 'repository trait.' Think of it like a blueprint for how we'll interact with data from the outside. We'll get into the nitty-gritty in a bit, but just know that the repository trait will be brought to life by the infra layer. It's like a set of rules the infra layer will follow to play nice with the data.</p>
<pre><code class="lang-rust"><span class="hljs-meta">#[async_trait]</span>
<span class="hljs-keyword">pub</span> <span class="hljs-class"><span class="hljs-keyword">trait</span> <span class="hljs-title">ReceiptRepository</span></span>{
    <span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">fn</span> <span class="hljs-title">find_by_id</span></span>(&amp;<span class="hljs-keyword">self</span>, id: <span class="hljs-built_in">String</span>) -&gt; <span class="hljs-built_in">Result</span>&lt;Receipt, <span class="hljs-built_in">Box</span>&lt;<span class="hljs-keyword">dyn</span> Error&gt;&gt;;
    <span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">fn</span> <span class="hljs-title">insert</span></span>(&amp;<span class="hljs-keyword">self</span>, receipt : Receipt) -&gt; <span class="hljs-built_in">Result</span>&lt;(), <span class="hljs-built_in">Box</span>&lt;<span class="hljs-keyword">dyn</span> Error&gt;&gt;;
}
</code></pre>
<ul>
<li>Infra:</li>
</ul>
<p>The infra layer is like the backstage crew for our code. It's responsible for making sure all the tools the domain layer needs are ready to go. So, when the use cases (we'll get to those in a bit) need something from the domain, the infra layer is there to provide it from whereever (database, API etc.). Think of it as a helpful assistant that hands over everything the main show (our app) needs to run smoothly.</p>
<pre><code class="lang-rust">    <span class="hljs-keyword">pub</span> <span class="hljs-function"><span class="hljs-keyword">fn</span> <span class="hljs-title">provide_mongo_collection</span></span>(&amp;<span class="hljs-keyword">self</span>, collection : <span class="hljs-built_in">String</span>) -&gt; <span class="hljs-keyword">impl</span> ReceiptRepository + <span class="hljs-built_in">Send</span> + <span class="hljs-built_in">Sync</span> {
        MongoCollection::new(<span class="hljs-keyword">self</span>.mongo_client.collection::&lt;Receipt&gt;(&amp;collection))
    }
</code></pre>
<ul>
<li>Use Cases</li>
</ul>
<p>Alright, let's talk about the use case. This is where the actual brainpower is. Imagine it as the decision-maker of our app. It's got this 'execute' function that does the heavy lifting. This function knows all the moves to get data in and out of the database. How does it know? With the repositories we talked about earlier. They give it the secret codes it needs to work its magic. Think of the use case as the superhero of our app, making sure everything happens just the way it should.</p>
<pre><code class="lang-rust">    <span class="hljs-keyword">pub</span> <span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">fn</span> <span class="hljs-title">execute</span></span>(&amp;<span class="hljs-keyword">self</span>, id: <span class="hljs-built_in">String</span>) -&gt; <span class="hljs-built_in">Result</span>&lt;domain::Receipt, <span class="hljs-built_in">Box</span>&lt;<span class="hljs-keyword">dyn</span> std::error::Error&gt;&gt; {
        <span class="hljs-keyword">self</span>.collection.find_by_id(id).<span class="hljs-keyword">await</span>
    }
</code></pre>
<ul>
<li>Dependency Injection:</li>
</ul>
<p>This module holds all the essential stuff the app needs, like connections to the database or other APIs. It's like a treasure chest of tools. But It's also a matchmaker. When we need a specific job done (that's where the use cases come in), it finds the perfect worker from the infra team, hooks them up, and creates a dream team for that task. This super-smooth teamwork means our app's different parts don't get tangled up; this is what low coupling is. It's like having a well-organized team that can pass the ball flawlessly.</p>
<p>This is <code>FeatureContainer</code></p>
<pre><code class="lang-rust">    <span class="hljs-keyword">pub</span> <span class="hljs-function"><span class="hljs-keyword">fn</span> <span class="hljs-title">get_receipt</span></span>(&amp;<span class="hljs-keyword">self</span>, from : <span class="hljs-built_in">String</span>) -&gt; GetReceipt {
        GetReceipt::new(<span class="hljs-built_in">Box</span>::new(<span class="hljs-keyword">self</span>.infra_provider.provide_mongo_collection(from)))
    }
</code></pre>
<ul>
<li>Presentation :</li>
</ul>
<p>The presentation layer is our app's front door for customers. It's like the face of our project, the part you'll start up. This is where the magic begins! It's the one that fires up the Actix web server and sets the rules for how it talks to the outside world.</p>
<p>The error manager is like the translator that takes the messy technical stuff and turns it into plain English for the customer.</p>
<p>The presentation layer also keeps 'DTO' – that's short for Data Transfer Object. It's like the messenger that carries data back and forth with the client; it turns this data into the proper format that the layer underneath understands (DAO), like translating between two languages. This layer also holds an instance of <code>FeatureContainer</code>, with essential connections. This container's job is to ensure everyone has what they need, like a backstage organizer (what we talked about just above).</p>
<pre><code class="lang-rust"><span class="hljs-meta">#[get(<span class="hljs-meta-string">"/receipt/{id}"</span>)]</span>
<span class="hljs-keyword">pub</span> <span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">fn</span> <span class="hljs-title">get_receipt</span></span>(
    data: Data&lt;FeatureContainer&gt;,
    req: HttpRequest,
    id: web::Path&lt;<span class="hljs-built_in">String</span>&gt;,
) -&gt; <span class="hljs-built_in">Result</span>&lt;HttpResponse, ApiError&gt; {

    <span class="hljs-keyword">let</span> from = req.headers().get(<span class="hljs-string">"from"</span>).ok_or_else(|| {
        ApiError::new(
            <span class="hljs-string">"Please pass a correct header"</span>.to_string(),
            <span class="hljs-string">"There is no header from"</span>.to_string(),
            actix_web::http::StatusCode::BAD_REQUEST,
        )
    })?.to_str().map_err(|err| {
        ApiError::new(
            <span class="hljs-string">"Please pass a correct header"</span>.to_string(),
            err.to_string(),
            actix_web::http::StatusCode::BAD_REQUEST,
        )
    })?.to_string();

    <span class="hljs-keyword">match</span> data.get_receipt(from).execute(id.into_inner()).<span class="hljs-keyword">await</span> {
        <span class="hljs-literal">Ok</span>(receipt) =&gt; <span class="hljs-literal">Ok</span>(HttpResponse::<span class="hljs-literal">Ok</span>().json(ReceiptDto::from(receipt))),
        <span class="hljs-literal">Err</span>(err) =&gt; <span class="hljs-literal">Err</span>(ApiError::new(
            <span class="hljs-string">"Error inserting receipt"</span>.to_string(),
            err.to_string(),
            actix_web::http::StatusCode::INTERNAL_SERVER_ERROR,
        )),
    }
}
</code></pre>
<h2 id="heading-conclusion">Conclusion</h2>
<p>If you want to see the code more in-depth, don't hesitate to take a look at: <a target="_blank" href="https://github.com/mathias-vandaele/receipts_inserter_onion">https://github.com/mathias-vandaele/receipts_inserter_onion</a></p>
]]></content:encoded></item><item><title><![CDATA[Managing exceptions in WebFlux using functional endpoint]]></title><description><![CDATA[Working with web flux
I have been working a lot with Webflux for the past 6 months, I have designed new services for my customers, and am always striving to deliver the best.
For most of my applications, I like to use the functional endpoint instead ...]]></description><link>https://mathias-vandaele.dev/managing-exceptions-in-webflux-using-functional-endpoint</link><guid isPermaLink="true">https://mathias-vandaele.dev/managing-exceptions-in-webflux-using-functional-endpoint</guid><category><![CDATA[Functional Programming]]></category><category><![CDATA[error handling]]></category><category><![CDATA[EndpointManagement]]></category><category><![CDATA[Springboot]]></category><category><![CDATA[Webflux]]></category><dc:creator><![CDATA[Mathias Vandaele]]></dc:creator><pubDate>Mon, 08 May 2023 07:58:55 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1683532780142/878dfc3b-69f2-457b-806e-e3e27891df0f.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h3 id="heading-working-with-web-flux">Working with web flux</h3>
<p>I have been working a lot with Webflux for the past 6 months, I have designed new services for my customers, and am always striving to deliver the best.</p>
<p>For most of my applications, I like to use the functional endpoint instead of the classic <code>RestController</code> decorator. I can manipulate directly the server response and I feel more in control of what I am doing. The whole thing feels more concise, explicit and easy to document.</p>
<p>Using functional endpoints also means I have to manage all the exceptions that might be thrown from my app. Using the functionals endpoint, I can't use the classic <code>RestControllerAdvice</code> to manage those exceptions. I have also seen in some blog posts that using it is a "bad practice".</p>
<p>I needed to find a way to manage exceptions properly for the services I am designing.</p>
<p>I came across multiple articles advising on extending <code>AbstractErrorWebExceptionHandler</code> which does not sound the right way to do it.</p>
<h3 id="heading-how-to-manage-the-exceptions">How to manage the exceptions?</h3>
<p>First, let's build a classic WebFlux app using SpringInitializr.</p>
<p>I am creating an interface that we can call <code>Error</code> that will describe the way of returning the right error content to the client.</p>
<pre><code class="lang-java"><span class="hljs-keyword">public</span> <span class="hljs-class"><span class="hljs-keyword">interface</span> <span class="hljs-title">Error</span> </span>{

    <span class="hljs-function">Mono&lt;ServerResponse&gt; <span class="hljs-title">getAsServerResponse</span><span class="hljs-params">()</span></span>;
}
</code></pre>
<p>Then I will Build my Exception Type, let's say I want Business Exception and Technical Exception.</p>
<p>I will create them extending <code>RuntimeException</code> and implementing <code>Error</code></p>
<p>I will give them a constructor, and a function to build a response body associated to the type of error.</p>
<pre><code class="lang-java"><span class="hljs-keyword">public</span> <span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">BusinessException</span> <span class="hljs-keyword">extends</span> <span class="hljs-title">RuntimeException</span> <span class="hljs-keyword">implements</span> <span class="hljs-title">Error</span> </span>{

    <span class="hljs-keyword">private</span> <span class="hljs-keyword">final</span> Integer code;
    <span class="hljs-keyword">private</span> <span class="hljs-keyword">final</span> String message;
    <span class="hljs-keyword">private</span> <span class="hljs-keyword">final</span> String advice;

    <span class="hljs-function"><span class="hljs-keyword">public</span> <span class="hljs-title">BusinessException</span><span class="hljs-params">(Integer code, String message, String advice)</span> </span>{
        <span class="hljs-keyword">super</span>(message);
        <span class="hljs-keyword">this</span>.code = code;
        <span class="hljs-keyword">this</span>.message = message;
        <span class="hljs-keyword">this</span>.advice = advice;
    }

    <span class="hljs-meta">@Override</span>
    <span class="hljs-function"><span class="hljs-keyword">public</span> Mono&lt;ServerResponse&gt; <span class="hljs-title">getAsServerResponse</span><span class="hljs-params">()</span> </span>{
        <span class="hljs-keyword">return</span> ServerResponse.badRequest().bodyValue(<span class="hljs-keyword">this</span>.createBusinessErrorMessage());
    }

    <span class="hljs-function"><span class="hljs-keyword">protected</span> BusinessError <span class="hljs-title">createBusinessErrorMessage</span><span class="hljs-params">()</span> </span>{
        <span class="hljs-keyword">return</span> BusinessError.builder()
                .message(<span class="hljs-keyword">this</span>.message)
                .advice(<span class="hljs-keyword">this</span>.advice)
                .code(<span class="hljs-keyword">this</span>.code)
                .build();
    }

}
</code></pre>
<p>The Technical can be created the same way.</p>
<p>Now If I wanna create a new exception, I would just have to extend either <code>TechnicalException</code> or <code>BusinessException</code>. The getAsServerResponse can stay the same or I can override it individually depending on the response code I want to return.</p>
<h3 id="heading-testing-the-error-management">Testing the error management.</h3>
<pre><code class="lang-java">    <span class="hljs-meta">@Bean</span>
    <span class="hljs-function">RouterFunction&lt;ServerResponse&gt; <span class="hljs-title">getEmployeeByIdRoute</span><span class="hljs-params">()</span> </span>{
        <span class="hljs-keyword">return</span> route(GET(<span class="hljs-string">"/hello-world/{name}"</span>),
                helloController::execute);
    }
</code></pre>
<p>I create a simple endpoint in which I will throw an error.</p>
<p>And here is how I manage everything at the same time.</p>
<pre><code class="lang-java"> <span class="hljs-keyword">return</span> Mono
                .fromSupplier(() -&gt; <span class="hljs-keyword">new</span> DummyReturn(request.pathVariable(<span class="hljs-string">"name"</span>)))
                .flatMap( dummyReturn -&gt; ServerResponse.ok().bodyValue(dummyReturn))

                .onErrorResume(BusinessException.class, BusinessException::getAsServerResponse)
                .onErrorResume(TechnicalException.class, TechnicalException::getAsServerResponse);
    }
</code></pre>
<p>Now I will just throw an error depending on random/fixed parameters</p>
<pre><code class="lang-java"><span class="hljs-function"><span class="hljs-keyword">public</span> <span class="hljs-title">DummyReturn</span><span class="hljs-params">(String name)</span> </span>{
        <span class="hljs-keyword">this</span>.name = name;
        <span class="hljs-keyword">this</span>.id = random.nextInt(<span class="hljs-number">20</span>);
        <span class="hljs-keyword">if</span> (Objects.equals(name, <span class="hljs-string">"error"</span>)) {
            <span class="hljs-keyword">throw</span> <span class="hljs-keyword">new</span> BusinessException(<span class="hljs-number">100</span>, <span class="hljs-string">"Error in DummyReturn constructor"</span>, <span class="hljs-string">"Enter a correct name"</span>);
        }
        <span class="hljs-keyword">if</span> (<span class="hljs-keyword">this</span>.id &gt; <span class="hljs-number">10</span>) {
            <span class="hljs-keyword">throw</span> <span class="hljs-keyword">new</span> TechnicalException(<span class="hljs-string">"An error happened while attributing the ID"</span>);
        }
    }
</code></pre>
<p>And here is the result</p>
<pre><code class="lang-http"><span class="hljs-attribute">GET http://localhost:8080/hello-world/df
HTTP/1.1 200 OK
Content-Type</span>: application/json
<span class="hljs-attribute">Content-Length</span>: 20

<span class="solidity">{
  <span class="hljs-string">"id"</span>: <span class="hljs-number">2</span>,
  <span class="hljs-string">"name"</span>: <span class="hljs-string">"df"</span>
}

GET http:<span class="hljs-comment">//localhost:8080/hello-world/df</span>
HTTP<span class="hljs-operator">/</span><span class="hljs-number">1.1</span> <span class="hljs-number">422</span> Unprocessable Entity
Content<span class="hljs-operator">-</span>Type: application<span class="hljs-operator">/</span>json
Content<span class="hljs-operator">-</span>Length: <span class="hljs-number">56</span>

{
  <span class="hljs-string">"message"</span>: <span class="hljs-string">"An error happened while attributing the ID"</span>
}</span>
</code></pre>
<p>I love this way cause we can be very explicit without having to write a ton of code.</p>
<p>If you are interested in seeing more of this, the entire codebase can be found on my GitHub at <a target="_blank" href="https://github.com/mathias-vandaele/webflux-error-management-functional-endpoint">https://github.com/mathias-vandaele/webflux-error-management-functional-endpoint</a></p>
<p>Thank you for reading!</p>
]]></content:encoded></item><item><title><![CDATA[Using Rust and Web assembly. Let's make heavy calculations on the client side.]]></title><description><![CDATA[We are witnessing the emergence of supercomputers and data centers capable of handling heavy calculations. With the rise of new applications that require serious computing power, such as artificial intelligence and mechanical computing, the need for ...]]></description><link>https://mathias-vandaele.dev/using-rust-and-web-assembly-lets-make-heavy-calculations-on-the-client-side</link><guid isPermaLink="true">https://mathias-vandaele.dev/using-rust-and-web-assembly-lets-make-heavy-calculations-on-the-client-side</guid><category><![CDATA[Rust]]></category><category><![CDATA[rust lang]]></category><category><![CDATA[distributed system]]></category><category><![CDATA[yew]]></category><category><![CDATA[WebAssembly]]></category><dc:creator><![CDATA[Mathias Vandaele]]></dc:creator><pubDate>Sun, 11 Dec 2022 12:01:44 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1683532842941/f67755c8-c731-438f-a7bd-f16d1c3de76e.webp" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>We are witnessing the emergence of supercomputers and data centers capable of handling heavy calculations. With the rise of new applications that require serious computing power, such as artificial intelligence and mechanical computing, the need for efficient computing solutions is more pressing than ever. RustLang has become one of my favorite languages after reading the book "The Rust Programming Language," and I am excited to explore its full potential.</p>
<p>I believe we can harness the power of Rust and WebAssembly to perform heavy calculations on the client-side. By creating a WebWorker in WebAssembly using Rust, we can achieve much greater processing power than traditional JavaScript-based solutions.</p>
<p>To demonstrate this, I plan to build a web application that calculates prime numbers. However, I want to ensure that the calculations are verified on the client-side, with the server managing connections and receiving results via an endpoint for the client.</p>
<p>To build this application, I will create a back end with Rust that will manage each client socket. The architecture I will be using is powerful, with the client socket being passed to the relevant function as needed. I will open a thread using a Tokio runtime to manage each client that wants to connect to the server.</p>
<pre><code class="lang-plaintext">fn handle_server(addr: String, sender: Sender&lt;Event&gt;) {
    Runtime::new().expect("Could not start a new runtime").block_on(async {
        let listener = TcpListener::bind(addr.to_string()).await.expect("Failed to bind socket");
        let mut id_manager = Counter::new(0);
        loop {
            match listener.accept().await {
                Ok((stream, socket_addr)) =&gt; {
                    tokio::spawn(handle_new_client(Connection::new(id_manager.next(),
                                                                   sender.clone(),
                                                                   stream,
                                                                   socket_addr)));
                }
                Err(e) =&gt; {
                    eprintln!("Impossible to connect with remote client, error : {}", e);
                }
            }
        }
    });
}
</code></pre>
<p>We give the task of managing the client a <code>Sender&lt;Event&gt;</code>, which is a channel that allows the task to communicate with the main thread.</p>
<pre><code class="lang-plaintext">async fn handle_new_client(connection: ImmatureConnection) {
    connection
        .accept_handshake().await
        .send_connection_opened_notification();
}
</code></pre>
<p>Once the handshake is complete, the <code>Connection</code> object has a method that sends a <code>connection_opened</code> notification using the channel.</p>
<p>Here's where it gets interesting: the channel sends the <code>Connection</code> object itself to the main thread wrapped in a notification.</p>
<pre><code class="lang-plaintext">fn send_connection_opened_notification(self) {
        self.event_sender.clone().send(Event::ConnectionOpened(self)).expect("Failed to send `Connection Opened` example");
    }
</code></pre>
<p>And all those notifications are managed in the main thread.</p>
<pre><code class="lang-plaintext">loop {
        let event = match sm.poll_event() {
            Ok(event) =&gt; { event }
            Err(e) =&gt; {
                eprintln!("An unexpected error happened when trying to poll an event : {}", e);
                exit(1);
            }
        };

        match event {
            Event::ConnectionOpened(connection) =&gt; {
                let next = job.pop_front().unwrap();
                id_worker_number_map.entry(connection.get_id()).or_insert(next);
                connection.send_message(next.to_string());
                job.push_back(number_manager.next())
            }
            Event::MessageReceived(connection, message) =&gt; {
                let value = id_worker_number_map.get_mut(&amp;connection.get_id()).expect("An unexpected error happened");
                if message == "true" {
                    //println!("worker n°{} has found a prime number : {}", connection.get_id(), value);
                    *last_prime.lock().unwrap() = *value
                }
                let next = job.pop_front().unwrap();
                connection.send_message(next.to_string());
                *value = next;
                job.push_back(number_manager.next());
            }
            Event::ConnectionClosed(connection) =&gt; {
                let value = id_worker_number_map.get(&amp;connection.get_id()).expect("An unexpected error happened");
                println!("Worker n°{} has closed the connection without finishing his job, his number {} goes back to the queue", connection.get_id(), value);
                job.push_back(*value);
                id_worker_number_map.remove(&amp;connection.get_id()).expect("This worker should have been in the map");
            }
            Event::CloseReceived(connection, _close_frame) =&gt; {
                let value = id_worker_number_map.get(&amp;connection.get_id()).expect("An unexpected error happened");
                println!("Worker n°{} has closed the connection without finishing his job, his number {} goes back to the queue", connection.get_id(), value);
                job.push_back(*value);
                id_worker_number_map.remove(&amp;connection.get_id()).expect("This worker should have been in the map");
            }
            e =&gt; println!("{:?}", e),
        }
    }
</code></pre>
<p>Now that the back-end is ready, I will be building the front-end in Yew. This will make it easy to compile Rust into a WebAssembly package that can be displayed on the front-end. To implement the WebWorker, I will be using Gloo, which is relatively easy to use.</p>
<p>After some work, I have created an agent that creates a web worker that connects to the back-end, requests a number, and replies with either true (if prime) or false (if not prime). I understand that this implementation is not optimized (it would be better to use a range of numbers where the client can send back the prime numbers in that range), but I created this to showcase the proof of concept of this architecture.</p>
<p>I used Traefik to manage the SSL certificate and other aspects efficiently, and now the application is ready to go! Check it out at <a target="_blank" href="https://distributed-computing.mathias-vandaele.dev/"><strong>https://distributed-computing.mathias-vandaele.dev/</strong></a></p>
<p>If you're interested, the entire codebase is available at <a target="_blank" href="https://github.com/mathias-vandaele/project-distributed-calculus"><strong>https://github.com/mathias-vandaele/project-distributed-calculus</strong></a>. Thank you for reading!</p>
]]></content:encoded></item><item><title><![CDATA[Why can't we predict Bitcoin's price ?]]></title><description><![CDATA[Even though I have never worked as a data engineer, I have been playing with machine learning for a while now.
At first, I was more interested in the mathematics behind it; I even made a repository where I tried implementing it.
Being in the world of...]]></description><link>https://mathias-vandaele.dev/why-cant-we-predict-bitcoins-price</link><guid isPermaLink="true">https://mathias-vandaele.dev/why-cant-we-predict-bitcoins-price</guid><category><![CDATA[Bitcoin]]></category><category><![CDATA[Python]]></category><category><![CDATA[Machine Learning]]></category><category><![CDATA[TensorFlow]]></category><category><![CDATA[LSTM]]></category><dc:creator><![CDATA[Mathias Vandaele]]></dc:creator><pubDate>Sun, 19 Jun 2022 16:14:12 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1655491052096/FIt0F73br.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Even though I have never worked as a data engineer, I have been playing with machine learning for a while now.
At first, I was more interested in the mathematics behind it; I even made a repository where I tried implementing it.</p>
<p>Being in the world of Bitcoin and, more generally, crypto-currencies for a while, I wondered if a complex algorithm could not be used to predict the price.</p>
<p>That happens to be <strong>harder</strong> than I thought. </p>
<p>After all the models I have done, I decided that I would try to prove the point that without very complex models, insight data, or a lot of money, you can't predict it; I came up to the conclusion that stock price and bitcoin price are just noise by itself. 
I believe it is possible to have more precise predictions, not just the price or the RSI.</p>
<p>How will I show you the difference between a predictable time series and a not-predictable one? </p>
<h1 id="heading-using-a-predictable-trigonometric-function">Using a predictable trigonometric function</h1>
<p>Here it is :</p>
<p>The predictable time series I am going to use is a sum of sinus and cosinus</p>
<pre><code>n = <span class="hljs-number">10000</span>

array = np.array([math.sin(i*<span class="hljs-number">0.02</span>) + math.cos(i*<span class="hljs-number">.05</span>) - math.sin(i*<span class="hljs-number">0.01</span>) <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(<span class="hljs-number">1</span>, n)])
fig, ax = plt.subplots()
ax.plot([i <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(<span class="hljs-number">1</span>, n)], array, linewidth=<span class="hljs-number">0.75</span>)
plt.show()
</code></pre><p>That gives us this curve :</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1655489774575/QiQstAK73.png" alt="image.png" /></p>
<p>Every algorithm you will find on the internet proposes you to predict the next value given the 150 last one. We are going to make something a bit different.
Let's take all our values and associate each of them with a buy index, which would be 1 when the best action to do is to buy and 0 when the best move would be to sell.</p>
<p>We can do this with a simple algorithm :</p>
<pre><code>SELL_INDEX = np.zeros((len(array), <span class="hljs-number">1</span>))

<span class="hljs-keyword">for</span> index, row <span class="hljs-keyword">in</span> enumerate(array):

    <span class="hljs-keyword">if</span> index &gt; len(array) - <span class="hljs-number">150</span>:
        <span class="hljs-keyword">continue</span>

    max_price = np.amax(array[index:index + <span class="hljs-number">150</span>])
    min_price = np.amin(array[index:index + <span class="hljs-number">150</span>])

    current_sell_index = (row - min_price) / (max_price - min_price)

    SELL_INDEX[index][<span class="hljs-number">0</span>] = <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> current_sell_index &gt; <span class="hljs-number">0.8</span> <span class="hljs-keyword">else</span> <span class="hljs-number">0</span>

data_with_sell_index = np.hstack((array.reshape(<span class="hljs-number">-1</span>,<span class="hljs-number">1</span>), SELL_INDEX))
data_final =  np.hstack( (data_with_sell_index,  np.arange(len(data_with_sell_index)).reshape(<span class="hljs-number">-1</span>, <span class="hljs-number">1</span>)) )
data_final = data_final[:len(data_final) - <span class="hljs-number">150</span>]
</code></pre><p>Let's apply it to our sum of sinus and cosinus curve, and that is what is showing :</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1655490178851/1GMD7wvVP.png" alt="image.png" /></p>
<p>Ok, we are all set now. The idea of the model now would be to predict the nᵗʰ buy index, using the 150 previous prices, so from n - 150 to n -1.</p>
<p>The model looks like this :</p>
<pre><code>input_layer = Input(shape=(<span class="hljs-number">150</span>, <span class="hljs-number">1</span>))
layer_1_lstm = LSTM(<span class="hljs-number">50</span>, return_sequences=True)(input_layer)
dropout_1 = Dropout(<span class="hljs-number">0.1</span>)(layer_1_lstm)
layer_2_lstm = LSTM(<span class="hljs-number">50</span>, return_sequences=True)(dropout_1)
dropout_2 = Dropout(<span class="hljs-number">0.1</span>)(layer_2_lstm)
layer_3_lstm = LSTM(<span class="hljs-number">50</span>)(dropout_2)

output_sell_index_proba = Dense(<span class="hljs-number">1</span>, activation=<span class="hljs-string">'sigmoid'</span>)(layer_3_lstm)

model = Model(inputs=input_layer, outputs=output_sell_index_proba)
model.compile(optimizer=<span class="hljs-string">'adam'</span>, loss=<span class="hljs-string">'binary_crossentropy'</span>, metrics=[keras.metrics.BinaryAccuracy()])
model.summary()
</code></pre><p>As I am getting either 1 or 0 as a result, I decided to use the metric <code>binaryAccuracy.</code>
and the loss function <code>binary_crossentropy.</code>
The optimizer, Adam is often the best choice; it is the one that has given me the best results.</p>
<p>After training with ten epochs, I end up with about <code>0.03</code> loss and almost <code>0.99</code> accuracy.</p>
<pre><code>Epoch: <span class="hljs-number">8.</span> Reducing Learning Rate <span class="hljs-keyword">from</span> <span class="hljs-number">0.0009227448608726263</span> to <span class="hljs-number">0.000913517433218658</span>
Epoch <span class="hljs-number">10</span>/<span class="hljs-number">10</span>
<span class="hljs-number">125</span>/<span class="hljs-number">125</span> [==============================] - <span class="hljs-number">28</span>s <span class="hljs-number">221</span>ms/step - loss: <span class="hljs-number">0.0389</span> - binary_accuracy: <span class="hljs-number">0.9853</span> - val_loss: <span class="hljs-number">0.0310</span> - val_binary_accuracy: <span class="hljs-number">0.9864</span>
</code></pre><p>Let's now try to predict from new data unseen by the algorithm during the training.</p>
<pre><code>data = np.array(data_final[:,<span class="hljs-number">0</span>][<span class="hljs-number">9000</span>:])
results = np.array([])
<span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range (<span class="hljs-number">150</span>, <span class="hljs-number">1000</span>):
    result = model.predict(data[i - <span class="hljs-number">150</span> : i].reshape(<span class="hljs-number">1</span>, <span class="hljs-number">150</span>, <span class="hljs-number">1</span>))
    results = np.append(result, results)
</code></pre><p>Here is the chart :</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1655645165144/SXY9l3xCF.png" alt="image.png" /></p>
<p>The accuracy of unseen data is pretty good, even though there are some inconsistencies.</p>
<h1 id="heading-using-bitcoin-price">Using Bitcoin price</h1>
<p>Let's plot the last 10000 CLOSE CANDLE on BTC (60S).</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1655653237319/-Xu1N84w3.png" alt="image.png" /></p>
<p>I will apply the previous algorithm I used for the trigonometric function, and here is the result.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1655653325382/Cfnjq5syw.png" alt="image.png" /></p>
<p>So now I have everything to work with, I can reapply the same process I did in the previous section. 
The only difference is that I will use a <code>MinMaxScaler</code>, so the input value will only vary from 0 to 1; neural networks have a hard time working with input that vary lot (here between 20k and 40k).</p>
<pre><code>scaler = MinMaxScaler(feature_range=(<span class="hljs-number">0</span>, <span class="hljs-number">1</span>))
fitter = scaler.fit(x)
x = fitter.transform(x)
</code></pre><p>The first difference that I noticed is the loss; It is stuck at 0.5</p>
<pre><code>Epoch: <span class="hljs-number">8.</span> Reducing Learning Rate <span class="hljs-keyword">from</span> <span class="hljs-number">0.000817907159216702</span> to <span class="hljs-number">0.0008097281097434461</span>
Epoch <span class="hljs-number">10</span>/<span class="hljs-number">10</span>
<span class="hljs-number">125</span>/<span class="hljs-number">125</span> [==============================] - <span class="hljs-number">27</span>s <span class="hljs-number">218</span>ms/step - loss: <span class="hljs-number">0.5074</span> - binary_accuracy: <span class="hljs-number">0.7952</span> - val_loss: <span class="hljs-number">0.4372</span> - val_binary_accuracy: <span class="hljs-number">0.8475</span>
</code></pre><p>And here is a prediction chart :</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1655654701569/A8ZCVvfrK.png" alt="image.png" /></p>
<p>There is nothing we can rely on.</p>
<p>The model does not find any pattern in the price; it is considered noise.</p>
<h1 id="heading-conclusion">Conclusion</h1>
<p>People have been trying to predict stock prices for the longest time; they have invented many  ways to do it : </p>
<ul>
<li>Technical analysis, like RSI, Ichimoku candle </li>
<li>Artificial inteligence technique</li>
<li>Sentiment analysis</li>
</ul>
<p>I think that no one can accurately predict the stock market without a solid understanding of the asset or massive investments.</p>
<p>You can find the code on my GitHub repository: https://github.com/mathias-vandaele/keras-research.</p>
]]></content:encoded></item></channel></rss>