<?xml version="1.0" encoding="UTF-8"?>
<rss  xmlns:atom="http://www.w3.org/2005/Atom" 
      xmlns:media="http://search.yahoo.com/mrss/" 
      xmlns:content="http://purl.org/rss/1.0/modules/content/" 
      xmlns:dc="http://purl.org/dc/elements/1.1/" 
      version="2.0">
<channel>
<title>Brian W. Lee</title>
<link>https://bl-ee.github.io/notes/</link>
<atom:link href="https://bl-ee.github.io/notes/index.xml" rel="self" type="application/rss+xml"/>
<description>Brian W. Lee&#39;s research homepage and notes.</description>
<generator>quarto-1.9.38</generator>
<lastBuildDate>Wed, 15 Jul 2026 00:00:00 GMT</lastBuildDate>
<item>
  <title>LP Duality from Separating Hyperplanes</title>
  <link>https://bl-ee.github.io/notes/lp-duality-separating-hyperplanes/</link>
  <description><![CDATA[ 




<p><em>I worked through the following derivation of LP duality to get some intuition for why admissible e-values for constrained testing problems take a certain affine form.</em></p>
<p>Consider the primal LP <img src="https://latex.codecogs.com/png.latex?%0Ap%5E%5Cstar%0A=%0A%5Cmax%20%5Cleft%5C%7B%0A%20%20c%5E%5Ctop%20x%0A%20%20:%0A%20%20x%20%5Cgeq%200,%5C;%0A%20%20Ax%20%5Cleq%20b%0A%5Cright%5C%7D,%0A"> where <img src="https://latex.codecogs.com/png.latex?A%20%5Cin%20%5Cmathbb%7BR%7D%5E%7Bm%20%5Ctimes%20n%7D">, <img src="https://latex.codecogs.com/png.latex?b%20%5Cin%20%5Cmathbb%7BR%7D%5Em">, and <img src="https://latex.codecogs.com/png.latex?c%20%5Cin%20%5Cmathbb%7BR%7D%5En">. Assume the primal is feasible and that <img src="https://latex.codecogs.com/png.latex?p%5E%5Cstar%20%3C%20%5Cinfty">. Then we know the dual is <img src="https://latex.codecogs.com/png.latex?%0A%5Cmin%20%5Cleft%5C%7B%0A%20%20b%5E%5Ctop%20%5Clambda%0A%20%20:%0A%20%20%5Clambda%20%5Cgeq%200,%5C;%0A%20%20A%5E%5Ctop%20%5Clambda%20%5Cgeq%20c%0A%5Cright%5C%7D.%0A"> Let’s derive this dual using a separating hyperplanes argument.</p>
<p>The first insight is to define the closed convex cone in a higher dimension, <img src="https://latex.codecogs.com/png.latex?%0AK%0A=%0A%5Cleft%5C%7B%0A%20%20(u,t)%20%5Cin%20%5Cmathbb%7BR%7D%5Em%20%5Ctimes%20%5Cmathbb%7BR%7D%0A%20%20:%0A%20%20%5Cexists%20x%20%5Cgeq%200,%5C;%0A%20%20Ax%20%5Cleq%20u,%5C;%0A%20%20t%20%5Cleq%20c%5E%5Ctop%20x%0A%5Cright%5C%7D.%0A"> A point <img src="https://latex.codecogs.com/png.latex?(u,t)%20%5Cin%20K"> means that using constraint budget <img src="https://latex.codecogs.com/png.latex?u">, one can achieve objective value at least <img src="https://latex.codecogs.com/png.latex?t">. To check that <img src="https://latex.codecogs.com/png.latex?K"> is a closed convex cone:</p>
<ol type="i">
<li><p><strong><img src="https://latex.codecogs.com/png.latex?K"> is a cone.</strong> If <img src="https://latex.codecogs.com/png.latex?(u,t)%20%5Cin%20K"> with witness <img src="https://latex.codecogs.com/png.latex?x">, then for any <img src="https://latex.codecogs.com/png.latex?%5Calpha%20%5Cgeq%200">, we have <img src="https://latex.codecogs.com/png.latex?A(%5Calpha%20x)%20%5Cleq%20%5Calpha%20u"> and <img src="https://latex.codecogs.com/png.latex?%5Calpha%20t%20%5Cleq%20c%5E%5Ctop(%5Calpha%20x)">.</p></li>
<li><p><strong><img src="https://latex.codecogs.com/png.latex?K"> is convex.</strong> If <img src="https://latex.codecogs.com/png.latex?(u_1,t_1),%20(u_2,t_2)%20%5Cin%20K"> with witnesses <img src="https://latex.codecogs.com/png.latex?x_1,x_2">, respectively, then for any <img src="https://latex.codecogs.com/png.latex?%5Ctheta%20%5Cin%20%5B0,1%5D">, <img src="https://latex.codecogs.com/png.latex?%0AA%5Cbigl(%5Ctheta%20x_1%20+%20(1-%5Ctheta)x_2%5Cbigr)%0A%5Cleq%0A%5Ctheta%20u_1%20+%20(1-%5Ctheta)u_2%0A"> and <img src="https://latex.codecogs.com/png.latex?%0A%5Ctheta%20t_1%20+%20(1-%5Ctheta)t_2%0A%5Cleq%0Ac%5E%5Ctop%5Cbigl(%5Ctheta%20x_1%20+%20(1-%5Ctheta)x_2%5Cbigr).%0A"></p></li>
<li><p><strong><img src="https://latex.codecogs.com/png.latex?K"> is closed.</strong> This is a little tricky to show, since the definition of <img src="https://latex.codecogs.com/png.latex?K"> leaves a witness <img src="https://latex.codecogs.com/png.latex?x"> implicit. We can view <img src="https://latex.codecogs.com/png.latex?K"> as a projection of a higher-dimensional polyhedral cone: <img src="https://latex.codecogs.com/png.latex?%0AK%0A=%0A%5Coperatorname%7Bproj%7D_%7Bu,t%7D%0A%5Cleft%5C%7B%0A%20%20(u,t,x)%0A%20%20:%0A%20%20x%20%5Cgeq%200,%5C;%0A%20%20Ax%20%5Cleq%20u,%5C;%0A%20%20t%20%5Cleq%20c%5E%5Ctop%20x%0A%5Cright%5C%7D.%0A"> Since the linear image of a polyhedron is a polyhedron, <img src="https://latex.codecogs.com/png.latex?K"> is a polyhedral cone, hence closed.</p></li>
</ol>
<p>The optimal value can be written as <img src="https://latex.codecogs.com/png.latex?%0Ap%5E%5Cstar%20=%20%5Csup%20%5Cleft%5C%7B%20t%20:%20(b,t)%20%5Cin%20K%20%5Cright%5C%7D,%0A"> and <img src="https://latex.codecogs.com/png.latex?(b,p%5E%5Cstar)"> is at the upper boundary of <img src="https://latex.codecogs.com/png.latex?K">. The second insight is to invoke a separating hyperplane argument at this point.</p>
<p>The strong separation result for closed convex cones says that if <img src="https://latex.codecogs.com/png.latex?C%20%5Csubseteq%20%5Cmathbb%7BR%7D%5Ed"> is a closed convex cone and <img src="https://latex.codecogs.com/png.latex?y%20%5Cnotin%20C">, then there exists <img src="https://latex.codecogs.com/png.latex?h%20%5Cin%20%5Cmathbb%7BR%7D%5Ed"> such that <img src="https://latex.codecogs.com/png.latex?h%5E%5Ctop%20z%20%5Cleq%200"> for all <img src="https://latex.codecogs.com/png.latex?z%20%5Cin%20C"> and <img src="https://latex.codecogs.com/png.latex?h%5E%5Ctop%20y%20%3E%200">. If <img src="https://latex.codecogs.com/png.latex?C"> is also polyhedral, we have the following stronger result: if <img src="https://latex.codecogs.com/png.latex?z_0%20%5Cin%20C"> and <img src="https://latex.codecogs.com/png.latex?e%20%5Cin%20%5Cmathbb%7BR%7D%5Ed"> are such that <img src="https://latex.codecogs.com/png.latex?z_0%20+%20%5Cvarepsilon%20e%20%5Cnotin%20C"> for all <img src="https://latex.codecogs.com/png.latex?%5Cvarepsilon%20%3E%200">, then there exists <img src="https://latex.codecogs.com/png.latex?h%20%5Cin%20%5Cmathbb%7BR%7D%5Ed"> such that <img src="https://latex.codecogs.com/png.latex?h%5E%5Ctop%20z%20%5Cleq%200"> for all <img src="https://latex.codecogs.com/png.latex?z%20%5Cin%20C">, <img src="https://latex.codecogs.com/png.latex?h%5E%5Ctop%20z_0%20=%200">, and <img src="https://latex.codecogs.com/png.latex?h%5E%5Ctop%20e%20%3E%200">. That is, the hyperplane <img src="https://latex.codecogs.com/png.latex?%5C%7Bz%20:%20h%5E%5Ctop%20z%20=%200%5C%7D"> supports <img src="https://latex.codecogs.com/png.latex?C"> at <img src="https://latex.codecogs.com/png.latex?z_0"> and separates <img src="https://latex.codecogs.com/png.latex?C"> from the ray <img src="https://latex.codecogs.com/png.latex?z_0%20+%20%5Cvarepsilon%20e"> with <img src="https://latex.codecogs.com/png.latex?%5Cvarepsilon%20%3E%200">.</p>
<p>Now let <img src="https://latex.codecogs.com/png.latex?z_0%20=%20(b,p%5E%5Cstar)%20%5Cin%20%5Cmathbb%7BR%7D%5E%7Bm+1%7D"> and <img src="https://latex.codecogs.com/png.latex?e%20=%20(0,%5Cldots,0,1)%20%5Cin%20%5Cmathbb%7BR%7D%5E%7Bm+1%7D">. For every <img src="https://latex.codecogs.com/png.latex?%5Cvarepsilon%20%3E%200">, <img src="https://latex.codecogs.com/png.latex?z_0%20+%20%5Cvarepsilon%20e%20%5Cnotin%20K">; otherwise, the definition of <img src="https://latex.codecogs.com/png.latex?p%5E%5Cstar"> would be contradicted. Hence, the polyhedral cone separation result gives a vector <img src="https://latex.codecogs.com/png.latex?h%20=%20(r,s)%20%5Cin%20%5Cmathbb%7BR%7D%5Em%20%5Ctimes%20%5Cmathbb%7BR%7D"> such that <img src="https://latex.codecogs.com/png.latex?%0Ar%5E%5Ctop%20u%20+%20st%20%5Cleq%200%0A%5Cquad%5Ctext%7Bfor%20all%20%7D%20(u,t)%20%5Cin%20K,%0A%5Cqquad%0Ar%5E%5Ctop%20b%20+%20sp%5E%5Cstar%20=%200,%0A%5Cqquad%0As%20%3E%200.%0A"> Now let <img src="https://latex.codecogs.com/png.latex?%5Clambda%20=%20-r/s">. Then <img src="https://latex.codecogs.com/png.latex?%0At%20%5Cleq%20%5Clambda%5E%5Ctop%20u%0A%5Cquad%5Ctext%7Bfor%20all%20%7D%20(u,t)%20%5Cin%20K,%0A%5Cqquad%0Ap%5E%5Cstar%20=%20%5Clambda%5E%5Ctop%20b.%0A"> This means the hyperplane <img src="https://latex.codecogs.com/png.latex?%5C%7B(u,t)%20:%20t%20=%20%5Clambda%5E%5Ctop%20u%5C%7D"> with normal vector <img src="https://latex.codecogs.com/png.latex?(-%5Clambda,1)"> supports <img src="https://latex.codecogs.com/png.latex?K"> at <img src="https://latex.codecogs.com/png.latex?(b,p%5E%5Cstar)">.</p>
<p>Notice that <img src="https://latex.codecogs.com/png.latex?%5Clambda"> is a dual-feasible solution. First, for any <img src="https://latex.codecogs.com/png.latex?v%20%5Cin%20%5Cmathbb%7BR%7D%5Em"> with <img src="https://latex.codecogs.com/png.latex?v%20%5Cgeq%200">, we have <img src="https://latex.codecogs.com/png.latex?(v,0)%20%5Cin%20K"> since <img src="https://latex.codecogs.com/png.latex?A0%20=%200%20%5Cleq%20v"> and <img src="https://latex.codecogs.com/png.latex?0%20%5Cleq%20c%5E%5Ctop%200%20=%200">. By the supporting inequality, <img src="https://latex.codecogs.com/png.latex?0%20%5Cleq%20%5Clambda%5E%5Ctop%20v"> must hold, which means <img src="https://latex.codecogs.com/png.latex?%5Clambda%20%5Cgeq%200">. Next, for every <img src="https://latex.codecogs.com/png.latex?x%20%5Cgeq%200">, <img src="https://latex.codecogs.com/png.latex?(Ax,c%5E%5Ctop%20x)%20%5Cin%20K"> since <img src="https://latex.codecogs.com/png.latex?Ax%20%5Cleq%20Ax"> and <img src="https://latex.codecogs.com/png.latex?c%5E%5Ctop%20x%20%5Cleq%20c%5E%5Ctop%20x">. Hence, <img src="https://latex.codecogs.com/png.latex?%0Ac%5E%5Ctop%20x%20%5Cleq%20%5Clambda%5E%5Ctop%20Ax,%0A"> or equivalently, <img src="https://latex.codecogs.com/png.latex?%0A(c-A%5E%5Ctop%5Clambda)%5E%5Ctop%20x%20%5Cleq%200.%0A"> Since this holds for all <img src="https://latex.codecogs.com/png.latex?x%20%5Cgeq%200">, we must have <img src="https://latex.codecogs.com/png.latex?c%20%5Cleq%20A%5E%5Ctop%20%5Clambda">. Hence, <img src="https://latex.codecogs.com/png.latex?%5Clambda%20%5Cgeq%200"> and <img src="https://latex.codecogs.com/png.latex?A%5E%5Ctop%20%5Clambda%20%5Cgeq%20c">, and by the supporting equality, <img src="https://latex.codecogs.com/png.latex?b%5E%5Ctop%20%5Clambda%20=%20p%5E%5Cstar">.</p>
<p>Combining this with weak duality, which is the easy direction, we conclude strong duality.</p>



 ]]></description>
  <guid>https://bl-ee.github.io/notes/lp-duality-separating-hyperplanes/</guid>
  <pubDate>Wed, 15 Jul 2026 00:00:00 GMT</pubDate>
</item>
</channel>
</rss>
