<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Tr8dr</title>
    <description>Musings on Algorithms, Models, and the Markets</description>
    <link>http://tr8dr.github.io/</link>
    <atom:link href="http://tr8dr.github.io//feed.xml" rel="self" type="application/rss+xml" />
    
      <item>
        <title>Arbitrage In DEFI (p2)</title>
        <description>&lt;p&gt;As mentioned in my prior post &lt;a href=&quot;https://tr8dr.github.io/MEV-p1/&quot;&gt;Arbitrage in DEFI (p1)&lt;/a&gt;, have been building and improving 
a MEV strategy in DEFI to perform both atomic and non-atomic arbitrage, backrunning, liquidations, etc.   In this article
we continue to focus on algorithms to detect and optimise arbitrage paths through the pool graph.&lt;/p&gt;

&lt;p&gt;Let us consider a simple graph of possible flows between 6 pools:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2024-11-16/graph1.png&quot; /&gt;&lt;/p&gt;

&lt;h2 id=&quot;bute-force-approach-for-small-graphs&quot;&gt;Bute-force approach (for small graphs)&lt;/h2&gt;
&lt;p&gt;For smaller graphs, we can avoid evaluating a complex optimisation by simply doing a DFS traversal to
determine all possible acyclic paths from source -&amp;gt; sink:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2024-11-16/paths.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Given all possible paths we can &lt;strong&gt;evaluate the optimal flow through each path&lt;/strong&gt; in order to maximize outcome.  As we put
more flow (size) through a given pool, we are penalized with an increasingly worse price and outgoing size.  Hence the
&lt;strong&gt;payout for a profitable path will be a concave function&lt;/strong&gt; such as this:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2024-11-16/profit.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;An optimisation on the path determines the size of the flow through the path that maximizes this function.  For a small
enough graph we could generate all such paths and maximize each one with a simple solver.  However, the
approach of &lt;strong&gt;generating all acyclic paths grows combinatorially and is not feasible&lt;/strong&gt; for a large graph.&lt;/p&gt;

&lt;h2 id=&quot;optimisation&quot;&gt;Optimisation&lt;/h2&gt;
&lt;p&gt;We want to find the weights through this graph such that the (outgoing - incoming) flow from source -&amp;gt; sink (our wallet) 
is maximized.  We can frame this problem graphically as a max-flow problem where we:&lt;/p&gt;

\[\begin{align*}
\text{maximize} \quad &amp;amp; \sum_{i \in N_{in}(sink)} \Lambda_i w_{i,sink} &amp;amp; \text{(flow into sink)} \\
\text{where:} \quad &amp;amp; \\
&amp;amp; \Lambda_i = g_i(\Delta_i) &amp;amp; \forall i \in N \text{ (node transfer functions)} \\
&amp;amp; \Delta_i = \sum_{j \in N_{in}(i)} \Lambda_j w_{j,i} &amp;amp; \forall i \in N \text{ (node input flows)} \\
&amp;amp; \Lambda_{src} = c &amp;amp; \text{(source capacity)} \\
\text{subject to:} \quad &amp;amp; \\
&amp;amp; \sum_{j \in N_{out}(i)} w_{i,j} \leq 1, \, w_{i,j} \in [0,1] &amp;amp; \forall i \in N \text{ (weight constraints)} \\
\text{where:} \quad &amp;amp; \\
&amp;amp; N \text{ is the set of all nodes} \\
&amp;amp; E \text{ is the set of all edges} \\
&amp;amp; N_{in}(i) \text{ is the set of nodes with edges into node } i \\
&amp;amp; N_{out}(i) \text{ is the set of nodes with edges from node } i \\
&amp;amp; g_i(\cdot) \text{ is the non-linear transfer function for node } i \\
&amp;amp; w_{i,j} \text{ is the weight of edge } (i,j) \\
&amp;amp; \Lambda_i \text{ is the output flow from node } i \\
&amp;amp; \Delta_i \text{ is the input flow to node } i \\
&amp;amp; c \text{ is the source node capacity}
\end{align*}\]

&lt;p&gt;Note that the &lt;strong&gt;node transfer function&lt;/strong&gt; (or amount of flow output for a given input) is as follows for a Uniswap V2-like
AMM:&lt;/p&gt;

\[g_i(\Delta) = r_{\Lambda} - \frac{k}{r_{\Delta} + \gamma \Delta}\]

&lt;h3 id=&quot;quasi-convex-optimisation&quot;&gt;Quasi-Convex Optimisation&lt;/h3&gt;
&lt;p&gt;In the paper &lt;a href=&quot;https://arxiv.org/pdf/2204.05238&quot;&gt;Optimal Routing for Constant Function Market Maker&lt;/a&gt;, Angeris et al
showed an elegant approach to determine the optimal amounts to be traded across a collection of pools maximizing some
utility function (for example maximize output of ETH) within the constraints of CFMMs.&lt;/p&gt;

\[\begin{align*}
\text{maximize} \quad &amp;amp; U(\Psi) &amp;amp; \text{(utility function)} \\
\text{subject to} \quad &amp;amp; \Psi = \sum_{i=1}^m A_i(\Lambda_i - \Delta_i) &amp;amp; \text{(trading function inputs } \Delta \text{, outputs } \Lambda \text{)}\\
&amp;amp; \varphi_i(R_i + \gamma_i\Delta_i - \Lambda_i) = \varphi_i(R_i), &amp;amp; \forall i \in m \text{ (constant product constraint)} \\
&amp;amp; \Delta_i \geq 0, \; \Lambda_i \geq 0 &amp;amp; \forall i \in m \text{ (traded amounts must be positive)}
\end{align*}\]

&lt;p&gt;where:&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;$(\Delta_i, \Lambda_i)$ is the amount in and amount out for the ith pool&lt;/li&gt;
  &lt;li&gt;$A_i$ indicates the adjacency matrix between tokens&lt;/li&gt;
  &lt;li&gt;$\phi_i(*)$ is an expression of the constant product reserve constraint&lt;/li&gt;
  &lt;li&gt;$U(\Psi)$ is the utility function, for example: $U(\Psi) = prices_{mkt} \times \Psi$, maximizing market value of 
net output&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The setup of this problem works beautifully on small to medium sized acyclic graphs, but is not stable or becomes overwhelmed in the following
contexts:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;graphs with cyclical feedback
    &lt;ul&gt;
      &lt;li&gt;the possible sub-graphs present different discontinuous gradient functions&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;large graphs with hundreds of thousands of edges
    &lt;ul&gt;
      &lt;li&gt;many of these algorithms have complexity $O(n^3 log(1/\epsilon))$, which does not scale for large sized
problems in the 10K+ range&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;We can make this approach workable if we can significantly reduce the graph, focusing on profitable neighborhoods, and 
remove cycles prior to applying the optimisation (I may elaborate on this in a subsequent post).&lt;/p&gt;

&lt;h3 id=&quot;max-flow-optimisation-for-acyclic-graphs&quot;&gt;Max-Flow Optimisation (for acyclic graphs)&lt;/h3&gt;
&lt;p&gt;If the graph is acyclic we can efficiently solve the max-flow problem as follows:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;determine a function to resolve flows given a weight vector (essentially a graph traversal)&lt;/li&gt;
  &lt;li&gt;determine the gradient of the network with respect to weights&lt;/li&gt;
  &lt;li&gt;determine objective function (for example maximize flow into sink node)&lt;/li&gt;
  &lt;li&gt;use L-BFGS-B to solve for the objective&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;This works well, but does require that the graph has no cycles.  With cycles a continuous gradient cannot be determined
due to the discontinuity of an edge / cycle being in the graph or excluded.  If the number of cycles is limited
can evaluate each graph alternative separately.  I have an implementation for this that arrives at the global optimum
in approximately $O(n\, log(n))$ time.&lt;/p&gt;

&lt;h3 id=&quot;heuristic-optimisation&quot;&gt;Heuristic Optimisation&lt;/h3&gt;
&lt;p&gt;As mentioned above convex optimisation approaches have issues in tackling the complexity and scale of these
graphs.  &lt;strong&gt;Heuristic optimisation&lt;/strong&gt; algorithms such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Differential_evolution&quot;&gt;differential evolution (DE)&lt;/a&gt; 
and &lt;a href=&quot;https://en.wikipedia.org/wiki/Simulated_annealing&quot;&gt;simulated annealing&lt;/a&gt; are well suited to finding a global optimum 
in the presence of:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;multiple local minima&lt;/li&gt;
  &lt;li&gt;discontinuous solution surfaces&lt;/li&gt;
  &lt;li&gt;mixed integer and continuous domain&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Neither of these solutions is guaranteed to arrive at the absolute maximum (minimum) due to their stochastic nature,
however can get quite close or alternatively converge arbitrarily close to the optimum with increased iteration.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Differential Evolution&lt;/strong&gt; is more capable than &lt;strong&gt;Simulated Annealing&lt;/strong&gt; in terms of being better able to explore the 
solution space.  This benefit comes at the cost of requiring more computation, at least in early iterations, however.
&lt;strong&gt;DE&lt;/strong&gt; is a special case of a &lt;strong&gt;genetic algorithm&lt;/strong&gt;, one where new individuals from generation to generation are expressed with gradient 
aware operators.  SA has some similarities to DE in the sense that the parameter vector is perturbed
in each successive generation, however one can easily get stuck in some part of the domain depending on the starting
point of the simulation.  DE is able to continue to explore the domain, though will tend to do so less in successive
generations.&lt;/p&gt;

&lt;p&gt;I found that &lt;strong&gt;differential evolution&lt;/strong&gt; was consistently able to arrive close to the global optimum, 
whereas simulated annealing might find a local optimum 50% of the time.  SA could be adjusted to
use a sampling (multi-simulation) approach and take the maximum over those simulations.&lt;/p&gt;

&lt;h3 id=&quot;bellman-ford-optimisation&quot;&gt;Bellman-Ford Optimisation&lt;/h3&gt;
&lt;p&gt;In the BF construction of the problem, we do not solve for max-flow directly, rather look for minimum paths through 
the graph where the weight is the $-log(price_i)$.  If we circuit from, say, WETH -&amp;gt; … -&amp;gt; WETH and have a negative
cumulative path, then have found a profitable path.&lt;/p&gt;

&lt;p&gt;The problem with BF algorithms are that they:&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;assume that price remains constant with size
    &lt;ul&gt;
      &lt;li&gt;so have to test for profitable paths on some unit size, later requiring a secondary optimisation to determine
appropriate size&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;algorithm cannot handle cycles
    &lt;ul&gt;
      &lt;li&gt;there are various heuristic approaches to trimming the graph in the presence of a cycle, but does not lead to
optimal solutions.  It can still provide viable, if not optimal, paths however.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;example&quot;&gt;Example&lt;/h2&gt;
&lt;p&gt;We want to find profitable paths through the following, given a source with 20 WETH:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Uniswap V2: ETH &amp;lt;-&amp;gt; USDT
    &lt;ul&gt;
      &lt;li&gt;reserves: 20100184, 10000, fee: 30bps&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Sushiswap V2: USDT &amp;lt;-&amp;gt; ETH
    &lt;ul&gt;
      &lt;li&gt;reserves: 20020000, 10830 fee: 30bps&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Uniswap V3: USDT &amp;lt;-&amp;gt; ETH
    &lt;ul&gt;
      &lt;li&gt;reserves: 20000000, 11040, fee: 30bps&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Shibaswap V2: USDT &amp;lt;-&amp;gt; ETH
    &lt;ul&gt;
      &lt;li&gt;reserves: 23000000, 10000, fee: 30bps&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Sushiswap V2 USDT &amp;lt;-&amp;gt; BTC
    &lt;ul&gt;
      &lt;li&gt;reserves: 101, 5000000, fee: 30bps&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Sushiswap V2: BTC &amp;lt;-&amp;gt; ETH
    &lt;ul&gt;
      &lt;li&gt;reserves: 100, 3518, fee: 30bps&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Here is the raw graph without optimisation and the graph post optimisation (where we have identified
the optimal flow through the graph from source -&amp;gt; sink):&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2024-11-16/raw-vs-optimised.png&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;solving-with-differential-evolution&quot;&gt;Solving with Differential Evolution&lt;/h3&gt;
&lt;p&gt;Differential Evolution is not the most efficient approach, however is one of the simplest and one that can achieve
global optimum.  In a subsequent post I will discuss a hybrid approach to reduce the size of the graph and allow for more economical
solutions.&lt;/p&gt;

&lt;p&gt;The approach to solving the max-flow problem with DE is as follows:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;determine a function to resolve flows given a weight vector
    &lt;ul&gt;
      &lt;li&gt;this is essentially a graph traversal where we evaluate incoming flows x weights and compute outgoing
flows&lt;/li&gt;
      &lt;li&gt;we also terminate and zero-out cycles when detected&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;write a function to enforce constraints $\sum_i w_i \le 1$ and also sparsify weight vectors
    &lt;ul&gt;
      &lt;li&gt;for sparsification I use $\bar{w}^k / \sum_i w_i^k $, pushing values towards 0 or 1&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;determine an objective function
    &lt;ul&gt;
      &lt;li&gt;maximize sum of incoming flows into sink node&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;write or use an existing differential evolution algorithm
    &lt;ul&gt;
      &lt;li&gt;python &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;scikit&lt;/code&gt; has a &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;differential_evolution&lt;/code&gt; function&lt;/li&gt;
      &lt;li&gt;rust has crates for this as well &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;differential_evolution&lt;/code&gt; and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;metaheuristics_nature&lt;/code&gt;&lt;/li&gt;
      &lt;li&gt;I’ve written a DE / GA implementation for C++ as well&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;h2 id=&quot;code&quot;&gt;Code&lt;/h2&gt;
&lt;p&gt;Realistically one will implement optimisation in Rust or C++, however for a more concise presentation am providing
some python code snippets here for the DE approach.&lt;/p&gt;

&lt;h3 id=&quot;pool&quot;&gt;Pool&lt;/h3&gt;
&lt;p&gt;This class defines functionality for a Uniswap V2-like pool or top-of-book V3 pools.  Here is the output function
defining amount out for a given amount in, according the CFMM constraints:&lt;/p&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;amount_out&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;amount_in&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;float&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&amp;gt;&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;float&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
    &lt;span class=&quot;s&quot;&gt;&quot;&quot;&quot;
    Compute the amount of money outgoing given incoming amount in the pool.  The amount in should be in
    the units of cin.
    &quot;&quot;&quot;&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;reserve0&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;gt;=&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;and&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;amount_in&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;k&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;reserve0&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;reserve1&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;gamma&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;1.0&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;fee&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;reserve1&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;k&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;/&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;reserve0&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;gamma&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;amount_in&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;else&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;amount_in&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h3 id=&quot;resolving-flows&quot;&gt;Resolving Flows&lt;/h3&gt;
&lt;p&gt;We want to determine flows through the graph, given a weight vector (where the weights define the % of flow per edge):&lt;/p&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;resolve_graph&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;weights&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;amount_in&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;):&lt;/span&gt;
    &lt;span class=&quot;s&quot;&gt;&quot;&quot;&quot;
    Resolve the graph for given weights and amount in
    &quot;&quot;&quot;&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;resolve_flow&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;weights&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;np&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;array&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;node&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Pool&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;seen&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;set&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;eps&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mf&quot;&gt;1e-4&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&amp;gt;&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;float&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;node&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;id&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;seen&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
            &lt;span class=&quot;k&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;node&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;outgoing&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;not&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;np&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;isnan&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;node&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;outgoing&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;else&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.0&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;not&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;np&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;isnan&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;node&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;outgoing&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;):&lt;/span&gt;
            &lt;span class=&quot;k&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;node&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;outgoing&lt;/span&gt;

        &lt;span class=&quot;n&quot;&gt;seen&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;add&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;node&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;id&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

        &lt;span class=&quot;c1&quot;&gt;# normalize incoming edges to sum &amp;lt;= 1 and sparsify
&lt;/span&gt;        &lt;span class=&quot;n&quot;&gt;incoming_edges&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;node&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;incoming_edges&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;G&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
        &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;normalize_and_sparsify&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;incoming_edges&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
        
        &lt;span class=&quot;c1&quot;&gt;# determine amount of inflow
&lt;/span&gt;        &lt;span class=&quot;n&quot;&gt;inflow&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.0&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;edge&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;incoming_edges&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
            &lt;span class=&quot;n&quot;&gt;iedge&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;edge_to_idx&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;edge&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;
            &lt;span class=&quot;n&quot;&gt;ancestor&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;pools&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;edge&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]]&lt;/span&gt;
            &lt;span class=&quot;n&quot;&gt;w&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;weights&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;iedge&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;
            &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;w&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;eps&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
                &lt;span class=&quot;n&quot;&gt;out&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;resolve_flow&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;weights&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;ancestor&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;seen&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
                &lt;span class=&quot;n&quot;&gt;inflow&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;w&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;out&lt;/span&gt;
            &lt;span class=&quot;k&quot;&gt;else&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
                &lt;span class=&quot;n&quot;&gt;out&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.0&lt;/span&gt;

            &lt;span class=&quot;c1&quot;&gt;# clamp near-zero weights to 0
&lt;/span&gt;            &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;out&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;==&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
                &lt;span class=&quot;n&quot;&gt;weights&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;iedge&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.0&lt;/span&gt;

        &lt;span class=&quot;n&quot;&gt;node&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;outgoing&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;node&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;amount_out&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;inflow&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;node&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;incoming&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;inflow&lt;/span&gt;

        &lt;span class=&quot;k&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;node&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;outgoing&lt;/span&gt;

    &lt;span class=&quot;n&quot;&gt;seen&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;set&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;([])&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;node&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;nodes&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;node&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;reset&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;node&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;nodes&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;resolve_flow&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;weights&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;node&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;seen&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h3 id=&quot;objective&quot;&gt;Objective&lt;/h3&gt;
&lt;p&gt;Our objective function is simple:&lt;/p&gt;
&lt;ol&gt;
  &lt;li&gt;resolve flows across graph for a given weight vector&lt;/li&gt;
  &lt;li&gt;determine output flow vs input flow&lt;/li&gt;
&lt;/ol&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;objective&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;weights&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;np&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;ndarray&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&amp;gt;&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;float&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
    &lt;span class=&quot;s&quot;&gt;&quot;&quot;&quot;
    Maximize profit as: `flow reaching sink` - `amount in` (as a minimization problem)
    &quot;&quot;&quot;&lt;/span&gt;
    &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;resolve_graph&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;weights&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;amount_in&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;pools&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;sink&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;].&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;incoming&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;amount_in&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h3 id=&quot;optimisation-1&quot;&gt;Optimisation&lt;/h3&gt;
&lt;p&gt;We then call our DE optimiser:&lt;/p&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;result&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;differential_evolution&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;func&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;objective&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;bounds&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;ranges&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;maxiter&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;maxiter&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;popsize&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;population&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;disp&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;True&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;weights&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;result&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;x&lt;/span&gt;
&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;profit&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;result&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;fun&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;amount_in&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;upcoming&quot;&gt;Upcoming&lt;/h2&gt;
&lt;p&gt;There is no entirely satisfactory solution for the optimisation problem in that:&lt;/p&gt;
&lt;ol&gt;
  &lt;li&gt;the presence of cycles invalidates a number of optimisation approaches&lt;/li&gt;
  &lt;li&gt;the scale of the problem is such that traditional optimisers cannot solve efficiently&lt;/li&gt;
  &lt;li&gt;heuristic optimisation can determine the global maximum +/- the stochastic nature of the algorithm&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;In a subsequent post will elaborate on an approach to reduce the problem significantly, making more amenable to
a variety of these approaches.&lt;/p&gt;
</description>
        <pubDate>Sat, 16 Nov 2024 11:00:00 +0000</pubDate>
        <link>http://tr8dr.github.io//MEV-p2/</link>
        <guid isPermaLink="true">http://tr8dr.github.io//MEV-p2/</guid>
      </item>
    
      <item>
        <title>Arbitrage In DEFI (p1)</title>
        <description>&lt;p&gt;I have been building and improving a MEV strategy in DEFI to perform both atomic and non-atomic arbitrage, backrunning, liquidations, etc.
In this post will focus on one of the hard algorithmic problems, namely, &lt;strong&gt;determining the optimal size and path of
arbitrage&lt;/strong&gt; through swap pools and other protocols.&lt;/p&gt;

&lt;p&gt;On Ethereum, for example, there are ~700K ERC20 tokens and a few hundred thousand AMM pools (fortunately only a fraction
of these pools and tokens are active).  We can consider the possible transactions and interactions across pools as
a &lt;strong&gt;directed graph&lt;/strong&gt;, where edges represent flows from a wallet or pool to another pool or wallet.   The size of this graph
is enormous, perhaps 200K nodes and a similar number of edges.   Here is a small example of such a graph:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2024-11-11/graph.png&quot; /&gt;&lt;/p&gt;

&lt;h2 id=&quot;the-problem&quot;&gt;The Problem&lt;/h2&gt;
&lt;p&gt;Given a source (our wallet) we want to &lt;strong&gt;determine if there are profitable arbitrage paths&lt;/strong&gt; through the graph.  Each node
represents a &lt;strong&gt;swap pool&lt;/strong&gt;, where the &lt;strong&gt;amount-in&lt;/strong&gt;, in one coin with result in some &lt;strong&gt;amount-out&lt;/strong&gt; in another coin.  This
presents itself as a &lt;strong&gt;max-flow&lt;/strong&gt; problem, where we want to:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;maximize the net amount out from source -&amp;gt; sink&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;i.e. amount into our wallet &amp;gt; amount out of our wallet at the end of the arbitrage paths&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;reject arbitrage paths if&lt;/strong&gt;:
    &lt;ul&gt;
      &lt;li&gt;expected cost of arbitrage &amp;gt; profit or minimum profit target, this includes gas cost &amp;amp; priority fee&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;AMMs&lt;/strong&gt; such as Uniswap V2 (and top of book V3) use the so-called “constant product” formulation for pricing $XY = k$, where X and Y are 
the reserves of coin 1 and coin 2.&lt;/p&gt;

&lt;p&gt;If we want to trade $\Delta X$ for some outgoing amount of coin $\Delta Y$, and assuming $\gamma = (1-fees)$, we arrive
at the relation:&lt;/p&gt;

\[(X + \gamma \Delta X) (Y - \Delta Y) = k\]

&lt;p&gt;expressing the notion that an incoming amount of $\Delta X$ is added to the X reserves and an outgoing amount of
$\Delta Y$ is removed from the Y reserves, all the while keeping $k$ constant.   Graphically, swapping adjusts size and
and price along a hyperbolic curve like this:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2024-11-11/AMM.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;We can formulate the problem, the &lt;strong&gt;max-flow problem&lt;/strong&gt;, by &lt;strong&gt;determining the weights&lt;/strong&gt; to assign to edges in the graph.  Let us
define the &lt;strong&gt;weights to be in [0,1]&lt;/strong&gt;, representing the % of coin to flow on a given edge from its immediate source node.
Given this setup, the following must be true:&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;the sum of weights for outgoing edges must be = 1&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;such that all outgoing flow from a node is handled&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;a weight on a given edge can be 0&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;indeed for a given arbitrage epoch, most weights will be 0 and just the pools and transfers active in the arbitrage
will have non-zero weights&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So, assuming we know how much size we are starting with at the source, it should just be a matter of determining the
weights through the graph such that we end with more size than we started with.&lt;/p&gt;

&lt;p&gt;If the pools were linear in terms of input -&amp;gt; output, this is a problem that could be solved with linear algebra, however 
most AMMs have a non-linear output function.  For any given node we could formulate its output based on the weighted
incoming flows:&lt;/p&gt;

\[\begin{align}
amount_{out} (F_i,w_i | i \in edges) &amp;amp;= Y - \frac{k}{X + \gamma \sum_{i \in edges} w_i F_i} \\
\sum_{i \in edges} w_i &amp;amp;= 1
\end{align}\]

&lt;p&gt;where \(F_i\) is the flow from the pool associated with the i’th edge and $w_i$ is the weight assigned to an incoming
edge.  One could create a chain of these equations linking nodes in the graph from source to sink.&lt;/p&gt;

&lt;h2 id=&quot;difficulties&quot;&gt;Difficulties&lt;/h2&gt;
&lt;p&gt;The graph of possible trades across pool presents problems:&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;all possible traversals of the graph &lt;strong&gt;has combinatorial complexity&lt;/strong&gt;.
    &lt;ul&gt;
      &lt;li&gt;the upper bound is on the order of O(n!), if fully connected&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;we encounter &lt;strong&gt;cycles&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;cycles pose problems for various max-flow or distance problems on a graph; for a max-flow problem we want to have
acyclic paths between the source and the sink.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;the amount of &lt;strong&gt;flow is non-linear with size&lt;/strong&gt;, as we have seen above
    &lt;ul&gt;
      &lt;li&gt;in fact Uniswap V2 clones present one of the easier cases for optimisation; Uniswap V3 presents an even more complex
discontinuous function from the point of view of optimisation.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;the-cycle-problem&quot;&gt;The cycle problem&lt;/h3&gt;
&lt;p&gt;A typical graph of AMM pools will have hundreds of cycles.  While we could do a DFS or BFS and eliminate cycles, the
DFS expansion would grow astronomically large.  Let’s consider a small example:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2024-11-11/problem-1.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The $w_2$ edge is reasonable if $w_1$ is disconnected ($w_1 = 0$) or vice-versa.  We can introduce indicator variables
$I_k \in {0,1}$, to gate flow avoiding cycles in the graph:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2024-11-11/problem-2.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;To determine where to put these indicator variables, we can do a &lt;strong&gt;breadth-first-search&lt;/strong&gt; (BFS) from the source down towards
the sink and make the following adjustments &lt;strong&gt;when a cycle is detected&lt;/strong&gt;:&lt;/p&gt;
&lt;ol&gt;
  &lt;li&gt;add indicator variable $I_j$ on the outgoing edge completing the cycle&lt;/li&gt;
  &lt;li&gt;add indicator variable $I_k$ on incoming edges already visited&lt;/li&gt;
  &lt;li&gt;add constraint $I_j + \sum_k I_k = 1$&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The above approach eliminates cycles and for proper construction of a &lt;strong&gt;max-flow&lt;/strong&gt; problem.&lt;/p&gt;

&lt;h3 id=&quot;the-optimisation-problem&quot;&gt;The optimisation problem&lt;/h3&gt;
&lt;p&gt;Optimisation is a challenge:&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;convex or quasi-convex optimisation&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;The bilinear $I_j w_j$ weight-indicator products on edges create non-convex constraints.  Unfortunately, this would 
require &lt;strong&gt;Mixed Integer Nonlinear Programming&lt;/strong&gt; to solve if done with a traditional optimiser.  If we get rid of the
binary indicator variables we could use a quasi-convex DQCP optimiser.&lt;/li&gt;
      &lt;li&gt;We could use a continuous function, albeit with cusp-like derivatives to model an indicator-like variable.  This might
not be very stable, however.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;size of problem&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;The size of the jacobian and other matrices used would be massive, making solving such a system in this way
impractical, even without the integer constraints.&lt;/li&gt;
      &lt;li&gt;there are other approaches that we will discuss that evade this issue.&lt;/li&gt;
      &lt;li&gt;there are also techniques we can use to dramatically reduce the size of the problem&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;We will discuss and evaluate these in further posts.&lt;/p&gt;

&lt;h2 id=&quot;upcoming&quot;&gt;Upcoming&lt;/h2&gt;
&lt;p&gt;We will evaluate the following in coming posts:&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;heuristic optimisation&lt;/li&gt;
  &lt;li&gt;convergence in DQCP with some reformulation of the problem&lt;/li&gt;
  &lt;li&gt;reduction of the problem (by a couple orders of magnitude)&lt;/li&gt;
&lt;/ul&gt;
</description>
        <pubDate>Mon, 11 Nov 2024 11:00:00 +0000</pubDate>
        <link>http://tr8dr.github.io//MEV-p1/</link>
        <guid isPermaLink="true">http://tr8dr.github.io//MEV-p1/</guid>
      </item>
    
      <item>
        <title>New Crypto Strategy</title>
        <description>&lt;p&gt;I have a new stat/arb strategy in crypto that uses an adaptive state based system to identify MR opportunities
on small portfolios of coins.  I have identified ~1600 such portfolios across 200 coins, each of which is traded
as a strategy.  In practice I trade a portfolio of these strategies (which I term, “stratlets”), where the strategy
is defined as a dynamically weighted portfolio of these stratlets.&lt;/p&gt;

&lt;h2 id=&quot;setup&quot;&gt;Setup&lt;/h2&gt;
&lt;p&gt;While each stratlet is adaptive, adjusting internal parameters over time, there are some hyper parameters for the
following layers:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;model&lt;/strong&gt;:
    &lt;ul&gt;
      &lt;li&gt;5 dynamic parameters inferred over time&lt;/li&gt;
      &lt;li&gt;1 fixed parameter&lt;/li&gt;
      &lt;li&gt;1 parameter to be optimised&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;trading state machine&lt;/strong&gt;:
    &lt;ul&gt;
      &lt;li&gt;3 parameters to be optimised&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;money management&lt;/strong&gt;:
    &lt;ul&gt;
      &lt;li&gt;2 parameters to be optimised&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In order to avoid overfitting, rather than optimising all 6 free parameters at once, took the following approach:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;optimise each layer separately:  i.e. the model, trading, and MM are optimised separately&lt;/li&gt;
  &lt;li&gt;restrict the parameter values to a small # of discrete possibilities&lt;/li&gt;
  &lt;li&gt;perturbation analysis to make sure the “solution” is not overly sensitive to the optimal setting&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;I assumed 30bps in transaction costs and slippage in backtesting.  Trade profitability is quite high, so is not terribly
sensitive to transaction cost.  I am expecting &amp;lt; 10bps in transaction costs, so the remaining 20bps relates to execution slippage.&lt;/p&gt;

&lt;h2 id=&quot;results&quot;&gt;Results&lt;/h2&gt;
&lt;p&gt;99% of the 1600 stratlets were &lt;strong&gt;very positive out-of-sample&lt;/strong&gt;.  The remaining ~15 losing stratlets had easily identified
issues during the in-sample period, with low to negative performance in-sample.  These 15 portfolios also scored poorly
in terms of stationarity as well.&lt;/p&gt;

&lt;p&gt;I observed a wide range of risk characteristics across the strategies (Sortino, Calmar, etc), so wanted to understand to
what degree the losing trades are correlated across stratlets.  &lt;strong&gt;If the losses were highly correlated&lt;/strong&gt; then bundling into a portfolio
would &lt;strong&gt;not tend to improve the overall risk&lt;/strong&gt;.  Fortunately, as we shall see, the losses are &lt;strong&gt;relatively uncorrelated&lt;/strong&gt;, leading to
&lt;strong&gt;improved risk characteristics&lt;/strong&gt; when trading a larger # of stratlets.&lt;/p&gt;

&lt;h3 id=&quot;small-portfolio&quot;&gt;Small Portfolio&lt;/h3&gt;
&lt;p&gt;Here is the performance of a small portfolio based on higher-liquidity stratlets ranked by the Calmar ratio within the validation period.
The individual stratlets have sortino’s ranking from 0.6 to 3.2 within this small set, however &lt;strong&gt;the effective Sortino for the
portfolio is higher&lt;/strong&gt;, at 4.3:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-09-01/top-5.png&quot; width=&quot;700&quot; height=&quot;550&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;medium-portfolio&quot;&gt;Medium Portfolio&lt;/h3&gt;
&lt;p&gt;The risk characteristics improve even further to 10.3 with the larger top-25 portfolio:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-09-01/top-25.png&quot; width=&quot;700&quot; height=&quot;550&quot; /&gt;&lt;/p&gt;

&lt;p&gt;So the good news is that the losses appear to be relatively independent across stratlets allowing us to average these out
with other profitable stratlets during the loss periods.&lt;/p&gt;

&lt;h3 id=&quot;large-portfolio&quot;&gt;Large Portfolio&lt;/h3&gt;
&lt;p&gt;I evaluated a portfolio based solely on the sortino seen within the training period, selecting stratlets with sortino &amp;gt; 3.
(selecting ~600 stratlets).  Note that the validation period was not used in portfolio
selection, providing a longer unseen period, from late March to current:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-09-01/all.png&quot; width=&quot;700&quot; height=&quot;550&quot; /&gt;&lt;/p&gt;

&lt;p&gt;This appears to an excellent result, with a Sortino of 22.4 and 4682% return.  However, the return seems &lt;strong&gt;too good to be true&lt;/strong&gt;,
and it is.  This result &lt;strong&gt;assumes&lt;/strong&gt; we can &lt;strong&gt;execute in size&lt;/strong&gt; (and at equal size) across stratlets.   In reality, scaling up
this strategy would underweight the majority of stratlets and overweight a smaller set with higher liquidity (the 
lower liquidity stratlets have higher returns than the highly liquid ones).&lt;/p&gt;

&lt;h2 id=&quot;notes&quot;&gt;Notes&lt;/h2&gt;
&lt;ol&gt;
  &lt;li&gt;I have not accounted for execution failures (for example missed legs)
    &lt;ul&gt;
      &lt;li&gt;will have to see how this plays out in live&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Many of these stratlets are liquidity constrained, so the actual return when size adjusted will be lower.
    &lt;ul&gt;
      &lt;li&gt;the lowest liquidity portfolios can only handle, maybe, 10K / trade, while the highest, potentially a few
million.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Live performance might degrade by 3 - 5x
    &lt;ul&gt;
      &lt;li&gt;due to the above considerations&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;The nature of stat/arb is that over time, the market will become more efficient
    &lt;ul&gt;
      &lt;li&gt;so I expect that the profitability of the strategy will diminish over time.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ol&gt;

</description>
        <pubDate>Wed, 01 Sep 2021 11:00:00 +0000</pubDate>
        <link>http://tr8dr.github.io//NewCryptoStrategy/</link>
        <guid isPermaLink="true">http://tr8dr.github.io//NewCryptoStrategy/</guid>
      </item>
    
      <item>
        <title>Crypto Trading Depth</title>
        <description>&lt;p&gt;I have a collection of crypto stat/arb strategies I plan to trade as a portfolio of strategies.  Each strategy trades a
small mean-reversion portfolio of loosely cointegrated coins, based on a bayesian state-based model.  The returns in 
cryptos for this sort of strategy are phenomenal, however, finding enough size can be difficult for some coin portfolios.&lt;/p&gt;

&lt;p&gt;In my universe of roughly 220 coins, there is significant variation in liquidity, requiring that I size these
strategy portfolios very differently depending on the coin composition.  Ultimately, the most robust answer re:
achievable scale lies in live-testing, however wanted to get some indicative view on liquidity apriori.&lt;/p&gt;

&lt;p&gt;Some relevant measures:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;&lt;strong&gt;spread-to-sweep&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;spread required to clear k$ in liquidity (with an aggressive order sweeping multiple levels of the order book).&lt;/li&gt;
      &lt;li&gt;one would rarely sweep the book to achieve a large $ amount unless there was an arbitrage, massive risk event,
or wanted to impact the market, however is useful in understanding book depth.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;maximum size achievable within k bps, and time period&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;Using trade data we can observe the $ amount that was filled within k bps of a reference price within
a given time period.&lt;/li&gt;
      &lt;li&gt;For different k bps buckets can produce a distribution of achieved size.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;h2 id=&quot;implementation&quot;&gt;Implementation&lt;/h2&gt;
&lt;p&gt;Many of the coins in my portfolio universe are traded on Binance and/or FTX.  Using Binance trade data observed the
following:&lt;/p&gt;

&lt;p&gt;Create &lt;strong&gt;distribution of size / price-spread bucket, conditional&lt;/strong&gt; on:&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;up, down momentum or none&lt;/li&gt;
  &lt;li&gt;low, medium, high volatility&lt;/li&gt;
  &lt;li&gt;day of week&lt;/li&gt;
  &lt;li&gt;time of day&lt;/li&gt;
  &lt;li&gt;aggressive or passive size&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The approach was to:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;choose a reference price on a sliding window (from a buyer or seller perspective)
    &lt;ul&gt;
      &lt;li&gt;we evaluate the size distribution for some forward period across the time series relative
to the price at the start of the window&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;observe the conditional variables (above) to classify volume&lt;/li&gt;
  &lt;li&gt;evaluate the size / spread cost distribution over periods: 5min, 10min, 15min .. 30min
    &lt;ul&gt;
      &lt;li&gt;replay trades to see what could be captured at various spread to base price levels&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;size by time-of-day, day-of-week&lt;/li&gt;
  &lt;li&gt;size differences as relates to volatilty and momentum&lt;/li&gt;
  &lt;li&gt;size achievable over different time periods&lt;/li&gt;
  &lt;li&gt;size achievable for a given spread cost&lt;/li&gt;
&lt;/ol&gt;

&lt;h2 id=&quot;time-of-day-and-day-of-week&quot;&gt;Time of Day and Day of Week&lt;/h2&gt;
&lt;p&gt;Dividing coins into the major (top 10% by volume), mid (above 50% &amp;lt; 90%), minor (&amp;lt; 50%), examining TOD and DOW patterns,
to answer the following questions:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;Is there a substantial difference in TOD / DOW profiles between top and bottom coin groups (no)&lt;/li&gt;
  &lt;li&gt;Is there a time-of-day effect (yes)&lt;/li&gt;
  &lt;li&gt;Is there a day-of-week effect (yes)&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-08-19/TOD.png&quot; width=&quot;600&quot; height=&quot;450&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-08-19/DOW.png&quot; width=&quot;600&quot; height=&quot;450&quot; /&gt;&lt;/p&gt;

&lt;h2 id=&quot;size--cost-distribution&quot;&gt;Size / Cost Distribution&lt;/h2&gt;
&lt;p&gt;I wanted to get a view of how much size is done (passively or aggressively) for a given direction over different time
periods and for some cost (in terms of bps deviation from the reference price).&lt;/p&gt;

&lt;p&gt;We can see a substantial difference in the amount of size traded within 5 minutes between BTC (in the top tier) and
BEAM (bottom tier):&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-08-19/BTC-size.png&quot; width=&quot;600&quot; height=&quot;450&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-08-19/BEAM-size.png&quot; width=&quot;600&quot; height=&quot;450&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;size-accumulation-by-time-in-market--cost-basis&quot;&gt;Size accumulation by time in market / cost basis&lt;/h3&gt;
&lt;p&gt;The size accumulated within 5min - 30min scaled linearly with the amount of time was in the market (for BTC/USDT):&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-08-19/scale-by-period.png&quot; width=&quot;300&quot; height=&quot;200&quot; /&gt;&lt;/p&gt;

&lt;p&gt;On average, the price hovered in the vicinity of the base price across the 5min - 30min period on
average, such that the volume achieved within 2bps grew 1:1 in proportion to the amount of time in the market.&lt;/p&gt;

&lt;h3 id=&quot;size-in-the-context-of-momentum&quot;&gt;Size in the context of momentum&lt;/h3&gt;
&lt;p&gt;Using the momentum labeler, observed the amount of size done in upward, downward, and neutral momentum (here are the stats
for BEAM/USDT, in the 2bps cost category):&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-08-19/BEAM-mv.png&quot; width=&quot;350&quot; height=&quot;250&quot; /&gt;&lt;/p&gt;

&lt;p&gt;As expected, the results show:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;Chasing momentum results in, overall, lower size
    &lt;ul&gt;
      &lt;li&gt;for example buying during upward momentum only accumulated 8k$ of size versus 11K$ neutral or 16K$ in downward momentum.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Buying in downward momentum allows for more net accumulation
    &lt;ul&gt;
      &lt;li&gt;needless to say, most size accumulated in this scenario can be passive&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;h3 id=&quot;size-in-the-context-of-different-volatility-regimes&quot;&gt;Size in the context of different volatility regimes&lt;/h3&gt;
&lt;p&gt;I divided volatility into low (&amp;lt;= 30% quantile), medium (30% &amp;lt; 70%), and high (&amp;gt; 70%) categories to see how this would
affect liquidity (again BEAM/USDT, in the 2bps category):&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-08-19/BEAM-vol.png&quot; width=&quot;350&quot; height=&quot;250&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The results show that there is much more liquidity in higher volatility regimes in the bottom ranked coins.  The effect is
less pronounced in the top ranked coins.&lt;/p&gt;

&lt;h2 id=&quot;size-by-coin-5mins&quot;&gt;Size By Coin (5mins)&lt;/h2&gt;
&lt;p&gt;Here is the average size seen over a 5min execution window by coin and cost bucket (on Binance):&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th style=&quot;text-align: right&quot;&gt;pair&lt;/th&gt;
      &lt;th style=&quot;text-align: right&quot;&gt;2bps&lt;/th&gt;
      &lt;th style=&quot;text-align: right&quot;&gt;5bps&lt;/th&gt;
      &lt;th style=&quot;text-align: right&quot;&gt;10bps&lt;/th&gt;
      &lt;th&gt;15bps&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;BTC/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;7.0e+06&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;7.8e+06&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;8.8e+06&lt;/td&gt;
      &lt;td&gt;9.7e+06&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;ETH/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;4.4e+06&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;4.7e+06&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;5.2e+06&lt;/td&gt;
      &lt;td&gt;5.7e+06&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;BNB/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;2.4e+06&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;2.6e+06&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;2.8e+06&lt;/td&gt;
      &lt;td&gt;3.0e+06&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;XRP/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.7e+06&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.8e+06&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.9e+06&lt;/td&gt;
      &lt;td&gt;2.1e+06&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;ADA/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.4e+06&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.4e+06&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.6e+06&lt;/td&gt;
      &lt;td&gt;1.7e+06&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;DOT/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;8.0e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;8.5e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;9.3e+05&lt;/td&gt;
      &lt;td&gt;9.9e+05&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;MATIC/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;7.6e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;7.9e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;8.4e+05&lt;/td&gt;
      &lt;td&gt;8.8e+05&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;LTC/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;5.6e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;6.0e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;6.6e+05&lt;/td&gt;
      &lt;td&gt;7.0e+05&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;ETC/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;5.1e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;5.3e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;5.6e+05&lt;/td&gt;
      &lt;td&gt;5.9e+05&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;EOS/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;5.0e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;5.3e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;5.7e+05&lt;/td&gt;
      &lt;td&gt;6.1e+05&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;VET/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;4.9e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;5.1e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;5.5e+05&lt;/td&gt;
      &lt;td&gt;5.8e+05&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;CHZ/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;4.9e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;5.1e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;5.4e+05&lt;/td&gt;
      &lt;td&gt;5.7e+05&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;LINK/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;4.4e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;4.7e+05&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;5.1e+05&lt;/td&gt;
      &lt;td&gt;5.5e+05&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;…&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;…&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;…&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;…&lt;/td&gt;
      &lt;td&gt;…&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;BEAM/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.2e+04&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.3e+04&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.3e+04&lt;/td&gt;
      &lt;td&gt;1.4e+04&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;WAN/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.2e+04&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.2e+04&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.3e+04&lt;/td&gt;
      &lt;td&gt;1.3e+04&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;KMD/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.1e+04&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.1e+04&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.2e+04&lt;/td&gt;
      &lt;td&gt;1.3e+04&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;NMR/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.1e+04&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.1e+04&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.2e+04&lt;/td&gt;
      &lt;td&gt;1.2e+04&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;TROY/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.0e+04&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.1e+04&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.1e+04&lt;/td&gt;
      &lt;td&gt;1.2e+04&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;REP/USDT&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.0e+04&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.0e+04&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;1.1e+04&lt;/td&gt;
      &lt;td&gt;1.2e+04&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;For those interested, happy to share the larger generated data set containing distribution by:  pair, side,
dow, tod, period, condition &amp;amp; level.&lt;/p&gt;

&lt;h2 id=&quot;excursions&quot;&gt;Excursions&lt;/h2&gt;
&lt;p&gt;The assumption in this size analysis is that we are executing (passively or aggressively) during some fixed period and
targetting some maximum entry cost (for example 2bps).  The price will undoubtedly drift outside of our target cost
region during the execution period, particularly if is a longer period or in the context of volatility or momentum.&lt;/p&gt;

&lt;p&gt;Hence was interested to know what the average maximum excursion (in bps) was for a given cost target.  For coins like BEAM,
this could be rather large depending on volatility context:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-08-19/BEAM-excursion.png&quot; width=&quot;600&quot; height=&quot;450&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-08-19/BTC-excursion.png&quot; width=&quot;600&quot; height=&quot;450&quot; /&gt;&lt;/p&gt;

&lt;h2 id=&quot;conclusions--notes&quot;&gt;Conclusions &amp;amp; Notes&lt;/h2&gt;
&lt;ol&gt;
  &lt;li&gt;time-of-day is material, with a 45% impact in terms of addressable volume&lt;/li&gt;
  &lt;li&gt;day-of-week variation is fairly marginal, with a 10% degradation over the weekend&lt;/li&gt;
  &lt;li&gt;addressable size in the top pairs is quite substantial, and in the bottom pairs quite marginal&lt;/li&gt;
  &lt;li&gt;some coins trade on multiple exchanges, increasing addressable size&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;I am not yet trading the referenced strategies, as need to set up an offshore entity or partner with a 3rd party.  Also, still
have some more work to do in terms of portfolio sizing and blending, but otherwise look very attractive.&lt;/p&gt;
</description>
        <pubDate>Fri, 20 Aug 2021 11:00:00 +0000</pubDate>
        <link>http://tr8dr.github.io//TradingDepth/</link>
        <guid isPermaLink="true">http://tr8dr.github.io//TradingDepth/</guid>
      </item>
    
      <item>
        <title>Pricing Deribit Options</title>
        <description>&lt;p&gt;We have been working on some option strategies and wanted to get a sense of how well BTC and ETH options are
priced on Deribit, i.e. is there a substantial IV premium over realized volatility or are options fairly priced.  At first glance, 
based on the documentation, it seemed that Deribit options were Europeans on spot or spot equivalent.  On 
closer inspection, however, and with some follow-ups with deribit, determined that this is actually a much
more complex product (to price).&lt;/p&gt;

&lt;p&gt;It turns out that the options have the following features:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;are &lt;strong&gt;“asian” options&lt;/strong&gt;, where the payout is based on the average of the underlier over some observation window
    &lt;ul&gt;
      &lt;li&gt;in particular these options use the average price over a 30 min or 5 min window in determining
the price at expiry&lt;/li&gt;
      &lt;li&gt;the average is an artithmetic average as opposed to a geometric average.  Asian option pricing for
arithmetic averages is more complicated as does not allow a closed-form solution.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;variable averaging window&lt;/strong&gt; depending on whether the expiry is matched 1:1 with a future
    &lt;ul&gt;
      &lt;li&gt;a 30 minute window is used for options where the expiry does not align exactly with a future&lt;/li&gt;
      &lt;li&gt;a 5 minute window is used for options where the expiry aligns with a future&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;The &lt;strong&gt;underlier is a future&lt;/strong&gt; or synthetic future, as opposed to spot
    &lt;ul&gt;
      &lt;li&gt;in practice, the future should converge to spot by expiry, but may have some different dynamics in between&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Our original goal was to back out the Black-Scholes equivalent implied volatilities (IV) and determine the volatility
surface. It is worth noting that Black Scholes (BS) has well known drawbacks (model error) in related to
its assumptions:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;constant volatility across the lifetime of the option&lt;/li&gt;
  &lt;li&gt;constant interest rate across the lifetime of the option&lt;/li&gt;
  &lt;li&gt;log normal price distribution (under estimates the tails)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In order to back out the BS IV for deribit options we will need to:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;work out how to price each “asian” option, given:
    &lt;ul&gt;
      &lt;li&gt;time to expiry, price of underlier, assumed risk-free-rate, volatility&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;interatively adjust volatility to determine an implied volatility that reprices to the premium
    &lt;ul&gt;
      &lt;li&gt;i.e.  f(spot price, time, rfr, iv) = observed premium&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;h2 id=&quot;pricing-an-asian-option&quot;&gt;Pricing an Asian Option&lt;/h2&gt;
&lt;p&gt;We cannot use the Black-Schole formula for European options to price an Asian Option, however we can borrow the
the price process assumed in BS. The Black-Scholes formulation of the price process evolves according to the
following SDE:&lt;/p&gt;

\[\partial{S_t} = rS_t\partial{t} + \sigma S_t \partial{W_t}\]

&lt;p&gt;where \(S_t\) is the price, \(r\) is the risk-free-rate, \(\sigma\) the volatility, \(W_t\) is a Weiner process.  We
will use this later in our solution for pricing an Asian option.&lt;/p&gt;

&lt;p&gt;The averaging period of an Asian option is &lt;strong&gt;path dependent&lt;/strong&gt;:  i.e. we care about where the price went across the averaging
period as opposed to the terminal price at expiry.  Due to this path dependency, while there are some closed form 
approximations, the most accurate way to evaluate these options is with a Monte Carlo (MC) technique.&lt;/p&gt;

&lt;p&gt;With the MC technique, we simulate many price paths to option expiry using the above Black-Scholes SDE.  We observe
the price at K points within the averaging period to determine the terminal price to be used for the option value at
expiry.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-07-28/paths.png&quot; width=&quot;800&quot; height=&quot;400&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Given, say, 100K paths, for each path \(i\) we compute the average \(A_i\) during the observation period and evaluate the payoff of
for a call option to be:&lt;/p&gt;

\[e^{-r t} \, max(A_i - K, 0)\]

&lt;p&gt;where \(K\) is the strike, \(r\) is the risk free rate, and \(A_i\) is the observed average on that path.   We
can determine the premium to be the &lt;strong&gt;expected value of the option payoff&lt;/strong&gt;, or in other words, the average payoff of
all of the paths.&lt;/p&gt;

\[premium = \frac{1}{n} \sum_{i = 1}^n e^{-r t} \, payoff \, (path_i)\]

&lt;h2 id=&quot;implementation&quot;&gt;Implementation&lt;/h2&gt;
&lt;p&gt;The first thing to note is that we need a discretized form of the BS price process.  Integrating for some \(\Delta t\),
this works out to be:&lt;/p&gt;

\[S_{t+\Delta t} = S_t e^{(r - \tfrac{1}{2} \sigma^2)\Delta t + \sigma \sqrt{\Delta t}W_t}\]

&lt;p&gt;Using the above we can use this to evolve a path for K time steps until expiry, expressing \(S_{t+\Delta t}\) in terms
of the prior price \(S_t\) recursively.&lt;/p&gt;

&lt;p&gt;We would like to limit the amount of computation required to evaluate a path. Noting that prior to the
averaging period, we have no price path dependency, we can avoid computing the path from \(t = 0\) until \(T - 30min\).
This can be accomplished by:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;sampling the “prior path” up to \(S_{T - 30m}\) from the BS forward price distribution
    &lt;ul&gt;
      &lt;li&gt;given by the above discretization, where \(\Delta t = T - 30m\)&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;using the sampled price \(S_{T - 30m}\), create a path with 30 additional prices
    &lt;ul&gt;
      &lt;li&gt;each price on the path post T - 30min will be used in the average for that particular path&lt;/li&gt;
      &lt;li&gt;we assume here, that the average is composed of 1min samples&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;There are other tricks we can deploy to reduce the computation time or improve accuracy, for example:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;choose an appropriate random number generator which has a distribution suitable for MC
    &lt;ul&gt;
      &lt;li&gt;Sobol and Halton sequences are excellent for MC&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;precompute the random normal samples and reuse in each calculation
    &lt;ul&gt;
      &lt;li&gt;the cost of generating random numbers on the fly can be substantial&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;antithetic variate
    &lt;ul&gt;
      &lt;li&gt;for every random draw can apply the + and - variation, creating 2 paths for every 1.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;variance reduction
    &lt;ul&gt;
      &lt;li&gt;reframe the variable (if possible) such that evaluating a problem with reduced variance, requiring fewer paths&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;h3 id=&quot;code&quot;&gt;Code&lt;/h3&gt;
&lt;p&gt;Here is a snippet of code for the simple case where we are pricing an asian option before the averaging
period has started:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;let W[t] be a precomputed cache of random normals drawn from a Sobol sequence&lt;/li&gt;
  &lt;li&gt;let t = time until maturity (in 1 / 365.25 units)&lt;/li&gt;
  &lt;li&gt;let spot = the underlier at the time of evaluation (the future price in this case)&lt;/li&gt;
  &lt;li&gt;let r = risk-free-rate (interest rate)&lt;/li&gt;
  &lt;li&gt;let dir = 1 = call, -1 = put&lt;/li&gt;
  &lt;li&gt;let sigma = volatility&lt;/li&gt;
&lt;/ul&gt;

&lt;div class=&quot;language-kotlin highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;kd&quot;&gt;val&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;T1min&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;1.0&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;/&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;mf&quot;&gt;365.25&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;24&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;60.0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;kd&quot;&gt;val&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;sqrtT1m&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;sqrt&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;nc&quot;&gt;T1min&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

&lt;span class=&quot;kd&quot;&gt;val&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;Tprior&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;t&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;averageperiod&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;nc&quot;&gt;T1min&lt;/span&gt;
&lt;span class=&quot;kd&quot;&gt;val&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;sqrtTp&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;sqrt&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;nc&quot;&gt;Tprior&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

&lt;span class=&quot;kd&quot;&gt;var&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;cpayoff&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.0&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;pathi&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;until&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;paths&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;/&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;kd&quot;&gt;val&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;ri&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;pathi&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;averageperiod&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;+&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;// compute 2 antithetic paths up to average measurement period: T - observation window&lt;/span&gt;
    &lt;span class=&quot;kd&quot;&gt;var&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;path1&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;spot&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;exp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;((&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;r&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.5&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sigma&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sigma&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;nc&quot;&gt;Tprior&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sigma&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sqrtTp&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;nc&quot;&gt;W&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;ri&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;])&lt;/span&gt;
    &lt;span class=&quot;kd&quot;&gt;var&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;path2&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;spot&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;exp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;((&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;r&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.5&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sigma&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sigma&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;nc&quot;&gt;Tprior&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sigma&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sqrtTp&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;nc&quot;&gt;W&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;ri&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;])&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;// compute average period for the 2 antithetic paths&lt;/span&gt;
    &lt;span class=&quot;kd&quot;&gt;var&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;cmean1&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.0&lt;/span&gt;
    &lt;span class=&quot;kd&quot;&gt;var&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;cmean2&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.0&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;i&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;..&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;averageperiod&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;path1&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;path1&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;exp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;((&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;r&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.5&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sigma&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sigma&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;nc&quot;&gt;T1min&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sigma&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sqrtT1m&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;nc&quot;&gt;W&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;ri&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;+&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;i&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;])&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;path2&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;path2&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;exp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;((&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;r&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.5&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sigma&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sigma&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;nc&quot;&gt;T1min&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sigma&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sqrtT1m&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;nc&quot;&gt;W&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;ri&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;+&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;i&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;])&lt;/span&gt;

        &lt;span class=&quot;n&quot;&gt;cmean1&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;+=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;path1&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;cmean2&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;+=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;path2&lt;/span&gt;
    &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;

    &lt;span class=&quot;c1&quot;&gt;// compute payoffs&lt;/span&gt;
    &lt;span class=&quot;kd&quot;&gt;val&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;mean1&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;cmean1&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;/&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;averageperiod&lt;/span&gt;
    &lt;span class=&quot;kd&quot;&gt;val&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;mean2&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;cmean2&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;/&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;averageperiod&lt;/span&gt;
    
    &lt;span class=&quot;n&quot;&gt;cpayoff&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;+=&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;max&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;dir&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;mean1&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;strike&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;),&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;cpayoff&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;+=&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;max&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;dir&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;mean2&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;strike&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;),&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;0.0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;kd&quot;&gt;val&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;df&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;nc&quot;&gt;Math&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;exp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(-&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;r&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;t&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;kd&quot;&gt;val&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;premium&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;df&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;cpayoff&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;/&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;paths&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;If the we are already in the averaging period then the code would be adjusted to combine the “fixings” (historical
price points) and the remaining path projected with MC.&lt;/p&gt;

&lt;h2 id=&quot;calculating-iv&quot;&gt;Calculating IV&lt;/h2&gt;
&lt;p&gt;Given the ability to, now, price an asian option, we can backout the value of sigma (\(\sigma\)), our implied
volatilty.  Making use of &lt;strong&gt;zbrent&lt;/strong&gt;, or another efficient root finder, can solve for:&lt;/p&gt;

\[pv(spot, time, rfr, sigma) - premium = 0\]

&lt;p&gt;for some range of sigma.&lt;/p&gt;

&lt;h2 id=&quot;notes&quot;&gt;Notes&lt;/h2&gt;
&lt;ul&gt;
  &lt;li&gt;deribit actually uses 6 second sampling rather than a 1min sampling during the average window
    &lt;ul&gt;
      &lt;li&gt;my current Deribit data is at 1min granularity, so assuming a different sampling frequency.&lt;/li&gt;
      &lt;li&gt;deribit does selectively use either 30min or 5min windows for the average as described above.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;deribit’s published IVs are priced assuming that the option is a European Black-Scholes
    &lt;ul&gt;
      &lt;li&gt;their IV is not correct, so use at your own risk&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;the option underlier may be a blend of the two straddling futures if there is no matching future&lt;/li&gt;
&lt;/ul&gt;

</description>
        <pubDate>Wed, 28 Jul 2021 11:00:00 +0000</pubDate>
        <link>http://tr8dr.github.io//AsianOptions/</link>
        <guid isPermaLink="true">http://tr8dr.github.io//AsianOptions/</guid>
      </item>
    
      <item>
        <title>Learning Candlestick Patterns</title>
        <description>&lt;p&gt;In the previous posts I described an Reinforcement Learning approach to “Learning the Exit” &lt;a href=&quot;https://tr8dr.github.io/RLp1/&quot;&gt;part 1&lt;/a&gt;,
&lt;a href=&quot;https://tr8dr.github.io/RLp2/&quot;&gt;part 2&lt;/a&gt;.  My initial conclusions there have been:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;reward smoothing&lt;/strong&gt; (with the labeler) &lt;strong&gt;leads to more robust results&lt;/strong&gt; than a reward on position exit
    &lt;ul&gt;
      &lt;li&gt;without smoothing the learning process struggled and had more volatility from epoch to epoch&lt;/li&gt;
      &lt;li&gt;obtained the best results with smoothed reward&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;obtained better results with &lt;strong&gt;discrete actions&lt;/strong&gt; (such as enter, hold, exit) rather than continuous
    &lt;ul&gt;
      &lt;li&gt;continuous action distributions appears to be much harder to learn, and the algorithms, such as DDPG or TD3
are much slower to evaluate than DQN or PPO2&lt;/li&gt;
      &lt;li&gt;PPO2 produced the best results, roughly 30% better than DQN&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;the &lt;strong&gt;results showed actionable positive P&amp;amp;L&lt;/strong&gt; after accounting for transaction costs and expected slippage
    &lt;ul&gt;
      &lt;li&gt;however did not get as close as I would like to the optimal reward in many scenarios&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In terms of improving RL performance, thought I could improve the information available to the agent by adding a feature
that classifies the current price-action in terms of:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;in &lt;strong&gt;momentum&lt;/strong&gt;&lt;/li&gt;
  &lt;li&gt;in &lt;strong&gt;trend&lt;/strong&gt;&lt;/li&gt;
  &lt;li&gt;momentum or trend &lt;strong&gt;at an end or turning point&lt;/strong&gt;&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;sideways price movement&lt;/strong&gt; or &lt;strong&gt;noise&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The goal for the RL strategy is to hold a position through a momentum or trend period in the direction of profit, and exiting 
when the features signal an end to momentum or a turning point in direction.   I provided the model with a number of features that 
assist in this decision, however did not have a feature encoding any information that might be present in the 
“chart”.  I am not a chartist, however there is undoubtedly information present in the price volume
activity as seen in a chart.&lt;/p&gt;

&lt;h2 id=&quot;a-possibility&quot;&gt;A Possibility&lt;/h2&gt;
&lt;p&gt;I came across an article &lt;a href=&quot;https://towardsdatascience.com/identifying-candlestick-patterns-using-deep-learning-b7d706726874&quot;&gt;Identifying Candlestick Patterns using Deep Learning&lt;/a&gt;,
which attempts to train a DL model to “become a chart reader”, effectively.  The aproach:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;use one of the top trained (million parameter) &lt;strong&gt;deep-learning vision models&lt;/strong&gt; as the basis&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;produced images of candlestick charts&lt;/strong&gt;, incrementing a rolling window across time&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;retrained the outer layers&lt;/strong&gt; to learn a new objective (&lt;strong&gt;transfer learning&lt;/strong&gt;) against these chart images&lt;/li&gt;
  &lt;li&gt;attempted to &lt;strong&gt;predict a fixed period forward return&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The use of &lt;strong&gt;transfer learning&lt;/strong&gt;, that of taking a trained
vision model, stripping off the final layers and retraining a new set of final layers, was interesting.&lt;/p&gt;

&lt;p&gt;However, I had the following reservations regarding the approach:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;complexity&lt;/strong&gt; and &lt;strong&gt;suitability&lt;/strong&gt; of a vision model with a million parameters
    &lt;ul&gt;
      &lt;li&gt;concerns about complexity adding to my trading environment&lt;/li&gt;
      &lt;li&gt;the chart domain is much simpler and regular than the objects these models are trained on, not ideally suited&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;using &lt;strong&gt;fixed period return labels is not robust&lt;/strong&gt;.
    &lt;ul&gt;
      &lt;li&gt;a positive or negative return could have real support or could just be noise&lt;/li&gt;
      &lt;li&gt;I found that t+5 returns often triggered a non-zero label due to apparent noise, and did not seem to
be part of a move or have prior support in the price or volume.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;a-better-approach&quot;&gt;A Better Approach&lt;/h2&gt;
&lt;p&gt;Rather than use an enormously complicated model, had the view that could create a simplified
representation of a “chart” and apply classification.  Like the above approach, I would present the prior K bars in some
form as features to a ML classifier, providing indication of individual and overall geometry.  Furthermore, I needed 
to address the labeling problem; avoiding noisy labels to the extent possible.&lt;/p&gt;

&lt;h3 id=&quot;labeling&quot;&gt;Labeling&lt;/h3&gt;
&lt;p&gt;In terms of labeling made use of a variation of our &lt;a href=&quot;https://tr8dr.github.io/labeling/&quot;&gt;labeler&lt;/a&gt; to label in the following 
manner:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;+2 / -2 for upward and downward momentum or trend with minimum magnitude of 3.5 x bar ATR&lt;/li&gt;
  &lt;li&gt;-1 / +1 for end of momentum or momentum direction transition periods&lt;/li&gt;
  &lt;li&gt;0 for sideways or noise&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-05-12/labeling.png&quot; width=&quot;600&quot; height=&quot;450&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;features&quot;&gt;Features&lt;/h3&gt;
&lt;p&gt;The next question was, what set of features captures the geometry of the chart pattern?  I started with simple
features, with the notion that could add more sophistication, as and if necessary:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;for each bar in our rolling window&lt;/strong&gt; (for example 20 bars):
    &lt;ul&gt;
      &lt;li&gt;normalized open to high&lt;/li&gt;
      &lt;li&gt;normalized open to low&lt;/li&gt;
      &lt;li&gt;normalized open to close&lt;/li&gt;
      &lt;li&gt;percentage change in volume relative to a rolling MA on volume&lt;/li&gt;
      &lt;li&gt;rolling cumulative return from start of window (this give the model a sense of geometry across the chart)&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;generate the above for 1min bars during market hours across a 10 year period&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;I did not experiment with other bar granularities, but expect that “fatter” bars would tend to produce higher
accuracy since they carry more aggregate information, and smooth out noise.  However, I need to balance between 
timeliness (bar frequency) versus potential gains in accuracy for longer periods.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Just as with the labeling, I used each asset’s ATR as a normalizer so that the same model could be applied across
a portfolio.&lt;/p&gt;

&lt;h3 id=&quot;results&quot;&gt;Results&lt;/h3&gt;
&lt;p&gt;Using a random forest classifier, obtained the following accuracies out-of-sample:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-05-12/oos-1.png&quot; width=&quot;500&quot; height=&quot;100&quot; /&gt;&lt;/p&gt;

&lt;p&gt;where the diagonal represents accuracies for labels { -2, -1, 0, +1, +2 } respectively.  I was surprised with the level
of accuracy given the rather simple features.   The transition period labels
{ -1, +1 } did especially well, which is useful.  I took the same model and applied it to other equitie and it gave
similar accuracy, which was reassuring.&lt;/p&gt;

&lt;p&gt;A friend of mine (thanks Steve W.), suggested that I take the prior bar prediction and feed that into the model as a 
new feature.  So for time \(t\), use the label predicted by the original model \(f(x_{t-1})\), augmenting the feature
set and training a new model \(f(x_t, f(x_{t-1}))\).  This improved the results significantly:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-05-12/oos-2.png&quot; width=&quot;500&quot; height=&quot;80&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;improving&lt;/strong&gt; from &lt;strong&gt;69%&lt;/strong&gt; on the directional labels to &lt;strong&gt;74%&lt;/strong&gt;, as well as improving on the neutral and transition labels.&lt;/p&gt;

&lt;h2 id=&quot;discussion&quot;&gt;Discussion&lt;/h2&gt;
&lt;p&gt;While the predictions are quite high, some thought is required in terms of how to use as a trading strategy in its
own right.  Correctly predicting the label for 1 min is not a return easily captured, and was not the intent of the
labeling.  The directional labels point to a bar belonging to a persistent market move, rather than being individually actionable. 
The transitional labels are likewise useful in identifying the end of market moves.&lt;/p&gt;

&lt;p&gt;The end goal was to produce another feature that would assist the RL algorithm in determining optimal exit strategy.  The 
signal can be used on its own however, taking into account the meaning assigned to these labels.&lt;/p&gt;

&lt;h3 id=&quot;possible-next-steps&quot;&gt;Possible Next Steps&lt;/h3&gt;
&lt;ul&gt;
  &lt;li&gt;produce classification for longer bars, augmenting the finer grain predictions&lt;/li&gt;
  &lt;li&gt;consider an exit approach without RL using this signal (to avoid the extra complexity of RL)&lt;/li&gt;
  &lt;li&gt;add as feature in RL and determine whether there is a significant advantage of RL over signal-based heuristics&lt;/li&gt;
  &lt;li&gt;evaluate the predictions as a trading strategy in its own right&lt;/li&gt;
&lt;/ul&gt;
</description>
        <pubDate>Wed, 12 May 2021 11:00:00 +0000</pubDate>
        <link>http://tr8dr.github.io//BarPatterns/</link>
        <guid isPermaLink="true">http://tr8dr.github.io//BarPatterns/</guid>
      </item>
    
      <item>
        <title>Learning the Exit (part 2)</title>
        <description>&lt;p&gt;As described in my prior post &lt;a href=&quot;https://tr8dr.github.io/RLp1/&quot;&gt;Learning the Exit (part 1)&lt;/a&gt;, I have a model that indicates 
mean reversion entries with ~81% accuracy, however I did not have a good approach in handling the exit.  While 81% of
MR signals had a minimum profit of 25% (of prior amplitude), the mean profit available was 150%, pointing to
a larger profit opportunity to be had if can better handle the exit.&lt;/p&gt;

&lt;p&gt;I have found it very difficult to predict the magnitude of the optimal MR returns a-priori (at the point of entry) in these 
scenarios, achieving a \(r^2\) of only 20% with various regression models.  Hence the approach calls for online 
decision making as to how to position and when to exit post entry.&lt;/p&gt;

&lt;p&gt;We want to determine a policy function \(\pi_{\theta}(a | s_t)\) to indicate the appropriate action to taken
for each bar post entry.  In order to learn this function, be it with a genetic algorithm or reinforcement learning, we 
need to define an objective/reward function.&lt;/p&gt;

&lt;h2 id=&quot;objectives&quot;&gt;Objectives&lt;/h2&gt;
&lt;p&gt;The reward function should include the following objectives:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;reward for proceeding on a profitable path&lt;/li&gt;
  &lt;li&gt;penalty for proceeding on an unprofitable path&lt;/li&gt;
  &lt;li&gt;penalty for non-performance in a prolonged sideways move&lt;/li&gt;
  &lt;li&gt;penalty for amount of risk assumed&lt;/li&gt;
  &lt;li&gt;transaction costs + slippage&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;a-reward-approach&quot;&gt;A reward approach&lt;/h2&gt;
&lt;p&gt;Many papers take the approach of &lt;strong&gt;deferring reward until position exit&lt;/strong&gt;, i.e. rewarding \(r_t = 0\) from \(T_{entry}\) to \(T_{exit-1}\) 
while in position, and only presenting the cumulative reward at \(T_{exit-1}\).  However, this approach leads to the &lt;strong&gt;credit assignment problem&lt;/strong&gt;,
where individual actions during the holding period have an unassigned reward, requiring the RL algorithm to approximate
the implied reward through some form of backward induction.&lt;/p&gt;

&lt;p&gt;We will &lt;strong&gt;avoid the credit assignment problem&lt;/strong&gt;, and hopefully motivate a faster and more accurate convergence towards a
policy model.  Providing continuous reward also should make the problem more amenable to gradient descent approaches.&lt;/p&gt;

&lt;p&gt;Making use of our &lt;a href=&quot;https://tr8dr.github.io/labeling/&quot;&gt;Labeler&lt;/a&gt;, we can label the cumulative return from the start of
the MR signaling \(T_{start}\) to some maximum \(T_{start} + \Delta T\).  Using these labels can determine the extent
of &lt;strong&gt;upward, downward, and sideways&lt;/strong&gt; price (return) movements.&lt;/p&gt;

&lt;h3 id=&quot;take-1&quot;&gt;Take 1&lt;/h3&gt;
&lt;p&gt;For each “bar” we will assign a reward based on the direction of the labeled extents:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;for upward sloping return movements: &lt;strong&gt;reward = dr/dt x position&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;note that dr/dt (slope) is positive, yielding a positive reward&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;for downward sloping return movements: &lt;strong&gt;reward = dr/dt x position&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;note that dr/dt (slope) is negative, yielding a negative reward&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;for sideways movements: &lt;strong&gt;reward = 0&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-04-27/base.png&quot; width=&quot;800&quot; height=&quot;600&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The cumulative reward in this scheme seems to be headed in the right direction, however takes considerable drawdown
risk (-39 bps) in order to achieve an additional 10bps.  I would like to see the optimum at the end of the 1st upward
section (in this case).&lt;/p&gt;

&lt;h2 id=&quot;take-2&quot;&gt;Take 2&lt;/h2&gt;
&lt;p&gt;The prior reward system did not add extra penalty for drawdown.  Without some additional penalty for drawdowns, the optimal reward
will be achieved at an end point with the maximum return, regardless of unrealized drawdown.  To ameliorate this we
use the following scheme:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;for upward sloping return movements: &lt;strong&gt;reward = dr/dt x position&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;as above&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;for drawdowns: &lt;strong&gt;reward = drawdown penalty x dr/dt x position&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;we add a penalty to discourage riding through drawdowns for minor eventual gains&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;for sideways movements: &lt;strong&gt;reward = sideways penalty x position&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;this acts as a discount factor, penalizing for non-performance during extended sideways moves&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Using a penalty of 2.0x for downward moves, the reward and cumulative looks like:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-04-27/modified.png&quot; width=&quot;800&quot; height=&quot;600&quot; /&gt;&lt;/p&gt;

&lt;p&gt;This is good, as now the optimum shows my preference for not riding through the -39bps of drawdown in order to achieve
an additional 10bps above the 50bp move of the first section.&lt;/p&gt;

&lt;h2 id=&quot;taking-this-further&quot;&gt;Taking this further&lt;/h2&gt;
&lt;p&gt;There are other aspects one may wish to incorporate into the reward function, for example:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;time:
    &lt;ul&gt;
      &lt;li&gt;do we discount rewards as time proceeds further away from signal inception?&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;transaction cost:
    &lt;ul&gt;
      &lt;li&gt;do we incorporate transaction cost at the point of entry or do we spread the cost across the
holding period?  Allocating the transaction cost to the reward at entry would tend to present a negative reward on
entry even when the entry is optimal (or likely to be profitable).&lt;/li&gt;
      &lt;li&gt;we could potentially ammortize the transaction cost across the remaining direction. If the agent does not hold for
the full extent of the direction, the remaining cost is allocated in the exit reward.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;we could also penalize for the amount of noise in a move, preferring less noisy moves.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Finally, would love to hear from readers regarding your own RL experience, problem setup, etc.&lt;/p&gt;

</description>
        <pubDate>Mon, 26 Apr 2021 11:00:00 +0000</pubDate>
        <link>http://tr8dr.github.io//RLp2/</link>
        <guid isPermaLink="true">http://tr8dr.github.io//RLp2/</guid>
      </item>
    
      <item>
        <title>Learning the Exit (part 1)</title>
        <description>&lt;p&gt;I have a model that indicates mean reversion with ~81% accuracy.  Here is an example of a MR scenario the model identified:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-04-26/MR.png&quot; width=&quot;500&quot; height=&quot;400&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The strategy is very good at detecting entry points (usually very close to optimal), however is fairly
crude in terms of how it determines the exit.  Given that 81% of MR (with this model) experiences a profit of at least 
25% of the prior amplitude, one could take the approach of waiting for a pullback of 25% less
stop-loss.&lt;/p&gt;

&lt;p&gt;While there are various profitable fixed stop-loss and profit target regimes for this model, using a fixed target
ignores market information post entry.  Here are some problems I want to solve:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;opportunity cost&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;The average profit available is ~150% of the prior amplitude (due to the scenario this model detects).&lt;/li&gt;
      &lt;li&gt;Selecting a conservative fixed exit target leaves “money on the table”&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;risk reduction&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;many of the risky scenarios involve a runaway trend post signal; this is detectable&lt;/li&gt;
      &lt;li&gt;blindly holding for a k% exit, ignoring information post entry, does not make sense&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;approaches&quot;&gt;Approaches&lt;/h2&gt;
&lt;p&gt;We might start with a labeling approach using the &lt;a href=&quot;https://tr8dr.github.io/labeling/&quot;&gt;Labeler&lt;/a&gt; to identity
the target mean-reversion extent.  The labels would indicate whether we should &lt;strong&gt;enter, build
position, reduce, or exit&lt;/strong&gt;.   Given the labels, we can attempt to &lt;strong&gt;construct a ML driven model&lt;/strong&gt; :&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;supervised classifier to predict individual labels&lt;/strong&gt; (not recommended)
    &lt;ul&gt;
      &lt;li&gt;This is unlikely to work without significant noise or lag.  The features may not have support for all movements or
periods in the movement.  For example some movements may be associated with significant directional volume
and others might not be.  Forcing a model to be able to predict all points in the timeseries is doomed to failure.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;learn a policy&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;Using either RL (Reinforcement Learning) or GP (Genetic Programming), learn a policy to decide what action to take at any point in time.
These algorithms are not (may not) be penalized for missing opportunities, rather can focus on learning detectable
scenarios.&lt;/li&gt;
      &lt;li&gt;we can make use of the labeler to inform the policy we are trying to learn; However, unlike classification, we are
not “forcing” the model to learn every label.  More on this when we discuss the reward function in the next post.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I will focus on &lt;strong&gt;learning a policy&lt;/strong&gt; in this post.&lt;/p&gt;

&lt;h2 id=&quot;application-of-rl&quot;&gt;Application of RL&lt;/h2&gt;
&lt;p&gt;There are many great papers and tutorials for reinforcement learning, so will not go into great detail here.  Our goal is 
to &lt;strong&gt;create a model that decides how to trade our MR exit&lt;/strong&gt;.  RL is framed as a markov process, where we progress
from one state \(s_t\) to the next \(s_{t+1}\), based on the action \(a_t\) taken by a “policy function” \(\pi_{\theta}(a | s_t)\).&lt;/p&gt;

&lt;p&gt;Given our entry signal, the policy function \(\pi_{\theta}(a | s_t)\) will &lt;strong&gt;propose an action \(a_t\) for each bar&lt;/strong&gt; post
MR signal, &lt;strong&gt;informing how to trade&lt;/strong&gt; the prospective MR entry and exit.  The actions it takes can either be discrete 
(for example Buy, Sell, None) or continuous (such as % allocation) in nature.&lt;/p&gt;

&lt;p&gt;We learn \(\pi_{\theta}(a | s_t)\) by &lt;strong&gt;training the policy function across many “episodes”&lt;/strong&gt;.  In our case an episode starts with
the MR signal at time \(T_{start}\) and ends within some maximum holding period \(T_{start} + \Delta T\).  Reinforcement
learning requires some form of feedback in order to effect learning, this is accomplished by assigning a reward
\(r(a_t,s_t)\) for each action given a state. The learning process attempts to learn a policy function that maximizes the accumulated 
reward across episodes (roughly speaking).&lt;/p&gt;

&lt;p&gt;For a given episode, &lt;strong&gt;there will be an optimal sequence of actions&lt;/strong&gt; \(a_{t = 0} .. a_{t=T}\) that &lt;strong&gt;maximizes the cumulative
reward function&lt;/strong&gt; \(Q(s_t,a_t)\).  In Q learning, this cumulative reward
function is expressed as (source &lt;a href=&quot;https://en.wikipedia.org/wiki/Q-learning&quot;&gt;wikipedia&lt;/a&gt;):&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-04-26/Q-learning.png&quot; width=&quot;700&quot; height=&quot;120&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Note that the above expression is only applicable to discrete action spaces.  Q-learning was an early implementation
of reinforcement learning, later superseded by a variety of more effective deep-learning based approaches, such as:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;A2C&lt;/li&gt;
  &lt;li&gt;DQN&lt;/li&gt;
  &lt;li&gt;DDPG&lt;/li&gt;
  &lt;li&gt;…&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;example&quot;&gt;Example&lt;/h3&gt;
&lt;p&gt;For example in the entry and exit scenario pictured here:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-04-26/MR.png&quot; width=&quot;500&quot; height=&quot;400&quot; /&gt;&lt;/p&gt;

&lt;p&gt;the optimal action sequence for an episode in our MR trading might be:&lt;/p&gt;

&lt;blockquote&gt;

  &lt;p&gt;{ \(a_0\) = None, .., \(a_9\) = None, \(a_{10}\) = &lt;strong&gt;Sell&lt;/strong&gt;, \(a_{11}\) = None, …, \(a_{80}\) = &lt;strong&gt;Buy&lt;/strong&gt; }&lt;/p&gt;

&lt;/blockquote&gt;

&lt;p&gt;In the above example:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;the (downward) MR signal was early by 10 bars (the market continued to climb for 10 bars); hence the
optimal action would be “no action” in the first ten bars.&lt;/li&gt;
  &lt;li&gt;the agent entered short at bar 10 and exited at bar 80.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Entering earlier would have reduced the cumulative reward, as an earlier entry would have resulted in negative reward for \(a_0\) through
\(a_10\). Likewise, entering later, say at \(a_20\), would have reduced the cumulative reward due to opportunity cost, not having
harvested the rewards from \(a_10\) to \(a_20\).&lt;/p&gt;

&lt;h2 id=&quot;sub-problems&quot;&gt;Sub-Problems&lt;/h2&gt;
&lt;p&gt;Breaking this down into sub-problems:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;determine feature set &amp;amp; state&lt;/strong&gt; likely to support policy learning
    &lt;ul&gt;
      &lt;li&gt;in addition to features want state indicating current position, P&amp;amp;L within episode, etc.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;defining a “gym”&lt;/strong&gt; where the agent trains on episodes (collection of MR entries and timeseries post prospective entry)
    &lt;ul&gt;
      &lt;li&gt;track current state given actions and environment&lt;/li&gt;
      &lt;li&gt;compute reward&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;determine &lt;strong&gt;reward approach&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;what is the best way to reward so that our training converges towards a model with desireable behavior?&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;determine &lt;strong&gt;actions&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;discrete or continuous action spaces: for example Buy, Sell, Hold (discrete) versus % allocation (continuous).&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I will focus on the reward function in the next post.&lt;/p&gt;
</description>
        <pubDate>Mon, 26 Apr 2021 11:00:00 +0000</pubDate>
        <link>http://tr8dr.github.io//RLp1/</link>
        <guid isPermaLink="true">http://tr8dr.github.io//RLp1/</guid>
      </item>
    
      <item>
        <title>Uniswap v3 &amp; Liquidity Provision</title>
        <description>&lt;p&gt;I had a look at Uniswap’s upcoming V3 protocol &lt;a href=&quot;https://uniswap.org/blog/uniswap-v3/&quot;&gt;overview&lt;/a&gt; and 
&lt;a href=&quot;https://uniswap.org/whitepaper-v3.pdf&quot;&gt;white paper&lt;/a&gt;.  There are some substantial improvements above the 
current V2, such as:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;ability to offer &lt;strong&gt;“concentrated” liquidity&lt;/strong&gt; (we’ll discuss this below)&lt;/li&gt;
  &lt;li&gt;ability to introduce pools with varying fee structure&lt;/li&gt;
  &lt;li&gt;advances in oracle efficiency and new oracle types&lt;/li&gt;
  &lt;li&gt;..&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For a market maker, the most important of these is the ability to “concentrate” liquidity.  What does this 
mean?  As of V2, one’s &lt;strong&gt;liquidity was uniformly distributed across all price levels&lt;/strong&gt;, as opposed to being distributed
near the prevailing price.  In limit-order book
terms, this would be equivalent to offering 1/k coins at each price level (given a maximum of K price levels).
This is &lt;strong&gt;very capital inefficient&lt;/strong&gt; as most trading occurs near the prevailing market price.&lt;/p&gt;

&lt;p&gt;Here is a diagram from the whitepaper illustrating the liquidity allocation of V2 versus possible distributions of
liquidity in the uniswap V3 pool:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-04-12/concentration.png&quot; width=&quot;800&quot; height=&quot;350&quot; /&gt;&lt;/p&gt;

&lt;p&gt;In V3, through the use of range position contracts, one can effect any distribution of liquidity desired.&lt;/p&gt;

&lt;h2 id=&quot;problems&quot;&gt;Problems&lt;/h2&gt;
&lt;p&gt;Being able to center liquidity around the mid price (or whatever price is advantageous for inventory) is an
indispensable tool for market makers (and yield farmers for that matter).   However as nice as this seems on the
surface, has some significant issues:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;transactions in the ethereum blockchain &lt;strong&gt;take 10 - 20 minutes to transact&lt;/strong&gt;&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;gas fees are expensive&lt;/strong&gt; and progressing higher due to the large # of ERC tokens on Ethereum&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A traditional &lt;strong&gt;market maker will adjust orders (liquidity positions) at high frequency&lt;/strong&gt; to remain competitive and keep
liquidity near the prevailing market price.  With a transaction overhead on the Ethereum blockchain measured in 10 - 20 
minutes, would require the market maker to &lt;strong&gt;spread liquidity across a wider price range&lt;/strong&gt; in order to maintain participation 
in the market, diluting capital efficiency and incurring more risk.&lt;/p&gt;

&lt;p&gt;The second issue relates to cost.  Centralized exchange, both traditional and crypto, do not charge fees for order
placement.  The gas fees inherent in updating the liquidity position range on Uniswap would be prohibitive for
traditional market making.&lt;/p&gt;

&lt;h2 id=&quot;a-solution-pegged-liquidity-contract&quot;&gt;A Solution: Pegged Liquidity Contract&lt;/h2&gt;
&lt;p&gt;Uniswap should introduce a &lt;strong&gt;Pegged Liquidity&lt;/strong&gt; order (contract).  The contract would indicate the following:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;allocate K coins&lt;/strong&gt; according to some distribution &lt;strong&gt;relative to the prevailing price&lt;/strong&gt;&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;express the distribution&lt;/strong&gt; as:
    &lt;ul&gt;
      &lt;li&gt;x coins at level price +/- offset1&lt;/li&gt;
      &lt;li&gt;y coins at level price +/- offset2&lt;/li&gt;
      &lt;li&gt;…&lt;/li&gt;
      &lt;li&gt;up to a sum of K coins&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;limit the lowest (highest) price&lt;/strong&gt; will offer on
    &lt;ul&gt;
      &lt;li&gt;this avoids following the price in a flash crash or just some level below which the market maker considers
uneconomical&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;As the prevailing market price moves (as determined by the oracle), the &lt;strong&gt;liquidity automatically moves with the price&lt;/strong&gt;.  This will
allow the market maker to participate in as many transactions as possible without having to issue a new directive
on the chain to move the liquidity position when the market moves.  This avoids both the ultra-high latency (10 - 20mins) and
the costs (gas) of moving the liquidity to meet the price.&lt;/p&gt;

&lt;p&gt;One can imagine other variations of this where the liquidity positioning only changes when the price increments 
by more than k%, keeping the liquidity/price offering constant for smaller moves.&lt;/p&gt;

&lt;p&gt;The proposed contract solves some of the concentration problem, however, it cannot go as far in offering the
granularity of control that is possible with centralized order books.  With a faster and cheaper blockchain
we can start approaching the efficiency of a central LOB.&lt;/p&gt;

&lt;h2 id=&quot;future-of-uniswap&quot;&gt;Future of Uniswap&lt;/h2&gt;
&lt;p&gt;Uniswap’s Achilles heel is that it is based on the Ethereum blockchain.  This has served it well until recently as many of the
coins traded on Uniswap are ERC-20 tokens, avoiding the need for Ethereum-transplanted proxy coins.  However, Ethereum
presents substantial problems for a DEX:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;non-deterministic transactions (required K blocks to verify transaction, much like bitcoin)
    &lt;ul&gt;
      &lt;li&gt;by non-deterministic mean that addition of a transaction to a block does not guarantee the transaction;
rather the transaction can only be assured after a certain number of blocks are added on the branch with
the transaction.&lt;/li&gt;
      &lt;li&gt;this increases latency substantially&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;scalability concerns (perhaps mitigated with side-chains)
    &lt;ul&gt;
      &lt;li&gt;however side-chains are not the most elegant solution and may present new attack vectors&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;high costs (gas fees)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The future for DEFI / DEX’s is likely to be on one of the newer blockchains, ones that:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;provide atomic transactions (ideally)&lt;/li&gt;
  &lt;li&gt;higher scalability&lt;/li&gt;
  &lt;li&gt;lower transaction costs&lt;/li&gt;
  &lt;li&gt;interop with other chains&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In this regard, &lt;strong&gt;Cosmos&lt;/strong&gt; looks quite attractive.  There are other blockchains and DEFI projects in competition
for the DEFI space (I will not enumerate them here).  The Ethereum Blockchain is innovating quickly (unlike the Bitcoin 
chain).  The momentum and the early lead of the Ethereum ecosystem could still win out in the end, in spite of the
attractive properties of these newer chains.&lt;/p&gt;

&lt;p&gt;Perhaps, though, the answer is not one L1 chain technology, but rather the projects that allow an &lt;strong&gt;inter&lt;/strong&gt;-net
of block chains - allow multiple chains to coexist and seemlessly interoperate.  &lt;strong&gt;Cosmos&lt;/strong&gt; is playing in this direction.&lt;/p&gt;

</description>
        <pubDate>Mon, 12 Apr 2021 11:00:00 +0000</pubDate>
        <link>http://tr8dr.github.io//PeggedLiquidity/</link>
        <guid isPermaLink="true">http://tr8dr.github.io//PeggedLiquidity/</guid>
      </item>
    
      <item>
        <title>Stable Coin Minting &amp; Momentum</title>
        <description>&lt;p&gt;A friend of mine (thanks Adal) suggested that I look at other stable coins, again observing the relationship between 
coin issuance and momentum.  Given that Circle (USDC) and Binance coin (BUSD) are widely traded, focused on these
two coins.&lt;/p&gt;

&lt;p&gt;Binance coin is newer than Circle, so I only have history for BUSD from late 2019.  However, will compare the signal
from these coins for events in 2020.&lt;/p&gt;

&lt;h2 id=&quot;2020-covid-bounce&quot;&gt;2020 Covid Bounce&lt;/h2&gt;
&lt;p&gt;Here is the combined signal for USDC and BUSD during the covid bounce (this looks like a better read than USDT):&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-01-09/covid.png&quot; width=&quot;800&quot; height=&quot;600&quot; /&gt;&lt;/p&gt;

&lt;h2 id=&quot;2020-q4-momentum&quot;&gt;2020 Q4 Momentum&lt;/h2&gt;
&lt;p&gt;USDC and BUSD accumulation is in advance of Q4 momentum:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/2021-01-09/Q4.png&quot; width=&quot;800&quot; height=&quot;600&quot; /&gt;&lt;/p&gt;

&lt;h2 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h2&gt;
&lt;p&gt;Analyzing issuance of stable coin gives us a view on buying interest.  A good signal would blend information from
both of these coins (perhaps looking at the cross-correlation of excitation).&lt;/p&gt;

</description>
        <pubDate>Sat, 09 Jan 2021 11:00:00 +0000</pubDate>
        <link>http://tr8dr.github.io//StableCoinMomentum/</link>
        <guid isPermaLink="true">http://tr8dr.github.io//StableCoinMomentum/</guid>
      </item>
    
  </channel>
</rss>