stsquad's blurbloghttps://stsquad.newsblur.com/2024-03-06T22:42:48.505000ZstsquadToday's only half of the leap year fun2024-03-06T22:42:48.505000Zhttps://rachelbythebay.com/w/2024/02/29/tib/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/todays-only-half-of-/2140417:cb7c5c">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/2140417.png" style="vertical-align: middle;width:16px;height:16px;"> Writing - rachelbythebay.</b>
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<div><p>
It's that time again, when code written in the past four years shows up
in our lives and breaks something. Still, while you're enjoying the
clown show of game companies telling people to manually set the clocks
on their consoles and people not being able to fill up their cars, keep
one thing in mind:
</p>
<p>
Only half of the fun of a leap year happens on February 29th.
</p>
<p>
The rest of it happens in ten months, when a bunch more code finds out
that it's somehow day 366, and promptly flips out. Thus, instead of
preparing to party, those people get to spend the day finding out why
their device is being stupid all of the sudden.
</p>
<p>
So, if you got through today unscathed, but are somehow counting days in
the year somewhere, you now have about 305 days to make sure you don't
have your own Zune bug buried in your own code.
</p>
<p>
...
</p>
<p>
One more random thought on the topic: some of today's kids will be
around to see what happens in 2100. That one will be all kinds of fun
to see who paid attention to their rules and who just guessed based on
a clean division by four.
</p></div>QEMU Advent Calendar 2018 is coming!2018-11-30T09:54:51.539000Zstefanha (noreply@blogger.com)http://blog.vmsplice.net/2018/11/qemu-advent-calendar-2018-is-coming.html<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/qemu-advent-calendar/845329:bf4b84">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/845329.png" style="vertical-align: middle;width:16px;height:16px;"> Stefan Hajnoczi.</b>
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p>QEMU Advent Calendar is running again this year. Each day from December 1st through 24th a surprise QEMU disk image will be released for your entertainment.</p> <p>Check out the website on December 1st for the first disk image:</p> <p><a href="https://www.qemu-advent-calendar.org/2018/">https://www.qemu-advent-calendar.org/2018/</a></p> <p>Thomas Huth is organizing QEMU Advent Calendar 2018 with the help of others from the QEMU community. If you want to contribute a disk image, take a look at the <a href="https://lists.gnu.org/archive/html/qemu-devel/2018-11/msg02388.html">call for images email</a>.</p>Team Chat2017-01-10T20:04:15.809000Zhttp://xkcd.com/1782/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/team-chat/5994357:ae5e6b">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/5994357.png" style="vertical-align: middle;width:16px;height:16px;"> xkcd.com:</b>
</td>
</tr>
<tr>
<td>
Truth
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<img alt="2078: He announces that he's finally making the jump from screen+irssi to tmux+weechat." src="http://imgs.xkcd.com/comics/team_chat.png" title="2078: He announces that he's finally making the jump from screen+irssi to tmux+weechat." />Chris Wellons: Some Performance Advantages of Lexical Scope2016-12-23T18:32:30.439000ZChristopher Wellonshttp://nullprogram.com/blog/2016/12/22/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/chris-wellons-some-p/1073:15a9f7">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/1073.png" style="vertical-align: middle;width:16px;height:16px;"> Planet Emacsen.</b>
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p>I recently had a discussion with <a href="http://ergoemacs.org/">Xah Lee</a> about lexical scope in
Emacs Lisp. The topic was why <code class="highlighter-rouge">lexical-binding</code> exists at a file-level
when there was already <code class="highlighter-rouge">lexical-let</code> (from <code class="highlighter-rouge">cl-lib</code>), prompted by my
previous article on <a href="http://nullprogram.com/blog/2016/12/11/">JIT byte-code compilation</a>. The specific
context is Emacs Lisp, but these concepts apply to language design in
general.</p>
<p>Until Emacs 24.1 (June 2012), Elisp only had dynamically scoped
variables — a feature, mostly by accident, common to old lisp
dialects. While dynamic scope has some selective uses, it’s widely
regarded as a mistake for local variables, and virtually no other
languages have adopted it.</p>
<p>Way back in 1993, Dave Gillespie’s deviously clever <code class="highlighter-rouge">lexical-let</code>
macro <a href="http://git.savannah.gnu.org/cgit/emacs.git/commit/?h=fcd73769&id=fcd737693e8e320acd70f91ec8e0728563244805">was committed</a> to the <code class="highlighter-rouge">cl</code> package, providing a rudimentary
form of opt-in lexical scope. The macro walks its body replacing local
variable names with guaranteed-unique gensym names: the exact same
technique used in macros to create “hygienic” bindings that aren’t
visible to the macro body. It essentially “fakes” lexical scope within
Elisp’s dynamic scope by preventing variable name collisions.</p>
<p>For example, here’s one of the consequences of dynamic scope.</p>
<div class="language-cl highlighter-rouge"><pre class="highlight"><code><span class="p">(</span><span class="nb">defun</span> <span class="nv">inner</span> <span class="p">()</span>
<span class="p">(</span><span class="k">setq</span> <span class="nv">v</span> <span class="ss">:inner</span><span class="p">))</span>
<span class="p">(</span><span class="nb">defun</span> <span class="nv">outer</span> <span class="p">()</span>
<span class="p">(</span><span class="k">let</span> <span class="p">((</span><span class="nv">v</span> <span class="ss">:outer</span><span class="p">))</span>
<span class="p">(</span><span class="nv">inner</span><span class="p">)</span>
<span class="nv">v</span><span class="p">))</span>
<span class="p">(</span><span class="nv">outer</span><span class="p">)</span>
<span class="c1">;; => :inner</span>
</code></pre>
</div>
<p>The “local” variable <code class="highlighter-rouge">v</code> in <code class="highlighter-rouge">outer</code> is visible to its callee, <code class="highlighter-rouge">inner</code>,
which can access and manipulate it. The meaning of the <em>free variable</em>
<code class="highlighter-rouge">v</code> in <code class="highlighter-rouge">inner</code> depends entirely on the run-time call stack. It might
be a global variable, or it might be a local variable for a caller,
direct or indirect.</p>
<p>Using <code class="highlighter-rouge">lexical-let</code> deconflicts these names, giving the effect of
lexical scope.</p>
<div class="language-cl highlighter-rouge"><pre class="highlight"><code><span class="p">(</span><span class="nb">defvar</span> <span class="nv">v</span><span class="p">)</span>
<span class="p">(</span><span class="nb">defun</span> <span class="nv">lexical-outer</span> <span class="p">()</span>
<span class="p">(</span><span class="nv">lexical-let</span> <span class="p">((</span><span class="nv">v</span> <span class="ss">:outer</span><span class="p">))</span>
<span class="p">(</span><span class="nv">inner</span><span class="p">)</span>
<span class="nv">v</span><span class="p">))</span>
<span class="p">(</span><span class="nv">lexical-outer</span><span class="p">)</span>
<span class="c1">;; => :outer</span>
</code></pre>
</div>
<p>But there’s more to lexical scope than this. Closures only make sense
in the context of lexical scope, and the most useful feature of
<code class="highlighter-rouge">lexical-let</code> is that lambda expressions evaluate to closures. The
macro implements this using a technique called <a href="https://en.wikipedia.org/wiki/Lambda_lifting"><em>closure
conversion</em></a>. Additional parameters are added to the original
lambda function, one for each lexical variable (and not just each
closed-over variable), and the whole thing is wrapped in <em>another</em>
lambda function that invokes the original lambda function with the
additional parameters filled with the closed-over variables — yes, the
variables (e.g. symbols) themselves, <em>not</em> just their values, (e.g.
pass-by-reference). The last point means different closures can
properly close over the same variables, and they can bind new values.</p>
<p>To roughly illustrate how this works, the first lambda expression
below, which closes over the lexical variables <code class="highlighter-rouge">x</code> and <code class="highlighter-rouge">y</code>, would be
converted into the latter by <code class="highlighter-rouge">lexical-let</code>. The <code class="highlighter-rouge">#:</code> is Elisp’s syntax
for uninterned variables. So <code class="highlighter-rouge">#:x</code> is <em>a</em> symbol <code class="highlighter-rouge">x</code>, but not <em>the</em>
symbol <code class="highlighter-rouge">x</code> (see <code class="highlighter-rouge">print-gensym</code>).</p>
<div class="language-cl highlighter-rouge"><pre class="highlight"><code><span class="c1">;; Before conversion:</span>
<span class="p">(</span><span class="k">lambda</span> <span class="p">()</span>
<span class="p">(</span><span class="nb">+</span> <span class="nv">x</span> <span class="nv">y</span><span class="p">))</span>
<span class="c1">;; After conversion:</span>
<span class="p">(</span><span class="k">lambda</span> <span class="p">(</span><span class="k">&rest</span> <span class="nv">args</span><span class="p">)</span>
<span class="p">(</span><span class="nb">apply</span> <span class="p">(</span><span class="k">lambda</span> <span class="p">(</span><span class="nv">x</span> <span class="nv">y</span><span class="p">)</span>
<span class="p">(</span><span class="nb">+</span> <span class="p">(</span><span class="nb">symbol-value</span> <span class="nv">x</span><span class="p">)</span>
<span class="p">(</span><span class="nb">symbol-value</span> <span class="nv">y</span><span class="p">)))</span>
<span class="o">'</span><span class="ss">#:x</span> <span class="o">'</span><span class="ss">#:y</span> <span class="nv">args</span><span class="p">))</span>
</code></pre>
</div>
<p>I’ve said on multiple occasions that <code class="highlighter-rouge">lexical-binding: t</code> has
significant advantages, both in performance and static analysis, and
so it should be used for all future Elisp code. The only reason it’s
not the default is because it breaks some old (badly written) code.
However, <strong><code class="highlighter-rouge">lexical-let</code> doesn’t realize any of these advantages</strong>! In
fact, it has worse performance than straightforward dynamic scope with
<code class="highlighter-rouge">let</code>.</p>
<ol>
<li>
<p>New symbol objects are allocated and initialized (<code class="highlighter-rouge">make-symbol</code>) on
each run-time evaluation, one per lexical variable.</p>
</li>
<li>
<p>Since it’s just faking it, <code class="highlighter-rouge">lexical-let</code> still uses dynamic
bindings, which are more expensive than lexical bindings. It varies
depending on the C compiler that built Emacs, but dynamic variable
accesses (opcode <code class="highlighter-rouge">varref</code>) take around 30% longer than lexical
variable accesses (opcode <code class="highlighter-rouge">stack-ref</code>). Assignment is far worse,
where dynamic variable assignment (<code class="highlighter-rouge">varset</code>) takes 650% longer than
lexical variable assignment (<code class="highlighter-rouge">stack-set</code>). How I measured all this
is a topic for another article.</p>
</li>
<li>
<p>The “lexical” variables are accessed using <code class="highlighter-rouge">symbol-value</code>, a full
function call, so they’re even slower than normal dynamic
variables.</p>
</li>
<li>
<p>Because converted lambda expressions are constructed dynamically at
run-time within the body of <code class="highlighter-rouge">lexical-let</code>, the resulting closure is
only partially byte-compiled even if the code as a whole has been
byte-compiled. In contrast, <code class="highlighter-rouge">lexical-binding: t</code> closures are fully
compiled. How this works is worth its own article.</p>
</li>
<li>
<p>Converted lambda expressions include the additional internal
function invocation, making them slower.</p>
</li>
</ol>
<p>While <code class="highlighter-rouge">lexical-let</code> is clever, and occasionally useful prior to Emacs
24, it may come at a hefty performance cost if evaluated frequently.
There’s no reason to use it anymore.</p>
<h3 id="constraints-on-code-generation">Constraints on code generation</h3>
<p>Another reason to be weary of dynamic scope is that it puts needless
constraints on the compiler, preventing a number of important
optimization opportunities. For example, consider the following
function, <code class="highlighter-rouge">bar</code>:</p>
<div class="language-cl highlighter-rouge"><pre class="highlight"><code><span class="p">(</span><span class="nb">defun</span> <span class="nv">bar</span> <span class="p">()</span>
<span class="p">(</span><span class="k">let</span> <span class="p">((</span><span class="nv">x</span> <span class="mi">1</span><span class="p">)</span>
<span class="p">(</span><span class="nv">y</span> <span class="mi">2</span><span class="p">))</span>
<span class="p">(</span><span class="nv">foo</span><span class="p">)</span>
<span class="p">(</span><span class="nb">+</span> <span class="nv">x</span> <span class="nv">y</span><span class="p">)))</span>
</code></pre>
</div>
<p>Byte-compile this function under dynamic scope (<code class="highlighter-rouge">lexical-binding:
nil</code>) and <a href="http://nullprogram.com/blog/2014/01/04/">disassemble it</a> to see what it looks like.</p>
<div class="language-cl highlighter-rouge"><pre class="highlight"><code><span class="p">(</span><span class="nv">byte-compile</span> <span class="nf">#'</span><span class="nv">bar</span><span class="p">)</span>
<span class="p">(</span><span class="nb">disassemble</span> <span class="nf">#'</span><span class="nv">bar</span><span class="p">)</span>
</code></pre>
</div>
<p>That pops up a buffer with the disassembly listing:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>0 constant 1
1 constant 2
2 varbind y
3 varbind x
4 constant foo
5 call 0
6 discard
7 varref x
8 varref y
9 plus
10 unbind 2
11 return
</code></pre>
</div>
<p>It’s 12 instructions, 5 of which deal with dynamic bindings. The
byte-compiler doesn’t always produce optimal byte-code, but this just
so happens to be <em>nearly</em> optimal byte-code. The <code class="highlighter-rouge">discard</code> (a very
fast instruction) isn’t necessary, but otherwise no more compiler
smarts can improve on this. Since the variables <code class="highlighter-rouge">x</code> and <code class="highlighter-rouge">y</code> are
visible to <code class="highlighter-rouge">foo</code>, they must be bound before the call and <a href="http://nullprogram.com/blog/2016/07/25/">loaded after
the call</a>. While generally this function will return 3, the
compiler cannot assume so since it ultimately depends on the behavior
<code class="highlighter-rouge">foo</code>. Its hands are tied.</p>
<p>Compare this to the lexical scope version (<code class="highlighter-rouge">lexical-binding: t</code>):</p>
<div class="highlighter-rouge"><pre class="highlight"><code>0 constant 1
1 constant 2
2 constant foo
3 call 0
4 discard
5 stack-ref 1
6 stack-ref 1
7 plus
8 return
</code></pre>
</div>
<p>It’s only 8 instructions, none of which are expensive dynamic variable
instructions. And this isn’t even close to the optimal byte-code. In
fact, as of Emacs 25.1 the byte-compiler often doesn’t produce the
optimal byte-code for lexical scope code and still needs some work.
<strong>Despite not firing on all cylinders, lexical scope still manages to
beat dynamic scope in performance benchmarks.</strong></p>
<p>Here’s the optimal byte-code, should the byte-compiler become smarter
someday:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>0 constant foo
1 call 0
2 constant 3
3 return
</code></pre>
</div>
<p>It’s down to 4 instructions due to computing the math operation at
compile time. Emacs’ byte-compiler only has rudimentary constant
folding, so it doesn’t notice that <code class="highlighter-rouge">x</code> and <code class="highlighter-rouge">y</code> are constants and
misses this optimization. I speculate this is due to its roots
compiling under dynamic scope. Since <code class="highlighter-rouge">x</code> and <code class="highlighter-rouge">y</code> are no longer exposed
to <code class="highlighter-rouge">foo</code>, the compiler has the opportunity to optimize them out of
existence. I haven’t measured it, but I would expect this to be
significantly faster than the dynamic scope version of this function.</p>
<h3 id="optional-dynamic-scope">Optional dynamic scope</h3>
<p>You might be thinking, “What if I really <em>do</em> want <code class="highlighter-rouge">x</code> and <code class="highlighter-rouge">y</code> to be
dynamically bound for <code class="highlighter-rouge">foo</code>?” This is often useful. Many of Emacs’ own
functions are designed to have certain variables dynamically bound
around them. For example, the print family of functions use the global
variable <code class="highlighter-rouge">standard-output</code> to determine where to send output by
default.</p>
<div class="language-cl highlighter-rouge"><pre class="highlight"><code><span class="p">(</span><span class="k">let</span> <span class="p">((</span><span class="nv">standard-output</span> <span class="p">(</span><span class="nv">current-buffer</span><span class="p">)))</span>
<span class="p">(</span><span class="nb">princ</span> <span class="s">"value = "</span><span class="p">)</span>
<span class="p">(</span><span class="nb">prin1</span> <span class="nv">value</span><span class="p">))</span>
</code></pre>
</div>
<p>Have no fear: <strong>With <code class="highlighter-rouge">lexical-binding: t</code> you can have your cake and
eat it too.</strong> Variables declared with <code class="highlighter-rouge">defvar</code>, <code class="highlighter-rouge">defconst</code>, or
<code class="highlighter-rouge">defvaralias</code> are marked as “special” with an internal bit flag
(<code class="highlighter-rouge">declared_special</code> in C). When the compiler detects one of these
variables (<code class="highlighter-rouge">special-variable-p</code>), it uses a classical dynamic binding.</p>
<p>Declaring both <code class="highlighter-rouge">x</code> and <code class="highlighter-rouge">y</code> as special restores the original semantics,
reverting <code class="highlighter-rouge">bar</code> back to its old byte-code definition (next time it’s
compiled, that is). But it would be poor form to mark <code class="highlighter-rouge">x</code> or <code class="highlighter-rouge">y</code> as
special: You’d de-optimize all code (compiled <em>after</em> the declaration)
anywhere in Emacs that uses these names. As a package author, only do
this with the namespace-prefixed variables that belong to you.</p>
<p>The only way to unmark a special variable is with the undocumented
function <code class="highlighter-rouge">internal-make-var-non-special</code>. I expected <code class="highlighter-rouge">makunbound</code> to
do this, but as of Emacs 25.1 it does not. This could possibly be
considered a bug.</p>
<h3 id="accidental-closures">Accidental closures</h3>
<p>I’ve said there are are absolutely no advantages to <code class="highlighter-rouge">lexical-binding:
nil</code>. It’s only the default for the sake of backwards-compatibility.
However, there <em>is</em> one case where <code class="highlighter-rouge">lexical-binding: t</code> introduces a
subtle issue that would otherwise not exist. Take this code for
example (and nevermind <code class="highlighter-rouge">prin1-to-string</code> for a moment):</p>
<div class="language-cl highlighter-rouge"><pre class="highlight"><code><span class="c1">;; -*- lexical-binding: t; -*-</span>
<span class="p">(</span><span class="nb">defun</span> <span class="nv">function-as-string</span> <span class="p">()</span>
<span class="p">(</span><span class="nv">with-temp-buffer</span>
<span class="p">(</span><span class="nb">prin1</span> <span class="p">(</span><span class="k">lambda</span> <span class="p">()</span> <span class="ss">:example</span><span class="p">)</span> <span class="p">(</span><span class="nv">current-buffer</span><span class="p">))</span>
<span class="p">(</span><span class="nv">buffer-string</span><span class="p">)))</span>
</code></pre>
</div>
<p>This creates and serializes a closure, which is <a href="http://nullprogram.com/blog/2013/12/30/">one of Elisp’s unique
features</a>. It doesn’t close over any variables, so it should be
pretty simple. However, this function will only work correctly under
<code class="highlighter-rouge">lexical-binding: t</code> when byte-compiled.</p>
<div class="language-cl highlighter-rouge"><pre class="highlight"><code><span class="p">(</span><span class="nv">function-as-string</span><span class="p">)</span>
<span class="c1">;; => "(closure ((temp-buffer . #<buffer *temp*>) t) nil :example)"</span>
</code></pre>
</div>
<p>The interpreter doesn’t analyze the closure, so just closes over
everything. This includes the hidden variable <code class="highlighter-rouge">temp-buffer</code> created by
the <code class="highlighter-rouge">with-temp-buffer</code> macro, resulting in an abstraction leak.
Buffers aren’t readable, so this will signal an error if an attempt is
made to read this function back into an s-expression. The
byte-compiler fixes this by noticing <code class="highlighter-rouge">temp-buffer</code> isn’t actually
closed over and so doesn’t include it in the closure, making it work
correctly.</p>
<p>Under <code class="highlighter-rouge">lexical-binding: nil</code> it works correctly either way:</p>
<div class="language-cl highlighter-rouge"><pre class="highlight"><code><span class="p">(</span><span class="nv">function-as-string</span><span class="p">)</span>
<span class="c1">;; -> "(lambda nil :example)"</span>
</code></pre>
</div>
<p>This may seem contrived — it’s certainly unlikely — but <a href="https://github.com/jwiegley/emacs-async/issues/17">it has come
up in practice</a>. Still, it’s no reason to avoid <code class="highlighter-rouge">lexical-binding: t</code>.</p>
<h3 id="use-lexical-scope-in-all-new-code">Use lexical scope in all new code</h3>
<p>As I’ve said again and again, always use <code class="highlighter-rouge">lexical-binding: t</code>. Use
dynamic variables judiciously. And <code class="highlighter-rouge">lexical-let</code> is no replacement. It
has virtually none of the benefits, performs <em>worse</em>, and it only
applies to <code class="highlighter-rouge">let</code>, not any of the other places bindings are created:
function parameters, <code class="highlighter-rouge">dotimes</code>, <code class="highlighter-rouge">dolist</code>, and <code class="highlighter-rouge">condition-case</code>.</p>Work2016-10-03T22:42:13.820000Zhttp://xkcd.com/1741/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/work/5994357:da30e1">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/5994357.png" style="vertical-align: middle;width:16px;height:16px;"> xkcd.com.</b>
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<img alt="Despite it being imaginary, I already have SUCH a strong opinion on the cord-switch firing incident." src="http://imgs.xkcd.com/comics/work.png" title="Despite it being imaginary, I already have SUCH a strong opinion on the cord-switch firing incident." />New zine: Linux debugging tools you'll love2016-09-10T09:58:56.443000Zhttp://jvns.ca/blog/2016/09/07/new-zine-linux-debugging-tools-youll-love/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/new-zine-linux-debug/3815987:cec409">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/3815987.png" style="vertical-align: middle;width:16px;height:16px;"> Julia Evans.</b>
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p>HELLO FRIENDS. I am announcing this everywhere because I'm very excited about
it. I released a new zine today! <a href="http://jvns.ca/debugging-zine.pdf">Read it here</a>! Read all my zine things at <a href="http://jvns.ca/zines/">jvns.ca/zines</a>!</p>
<p>This zine is about some of my favorite Linux debugging tools, especially tools that I don't think are as well-known as they should be. It covers <code>strace</code>, <code>opensnoop</code>/<code>eBPF</code>, and <code>dstat</code>! <code>netcat</code>, <code>netstat</code>, <code>tcpdump</code>, <code>wireshark</code>, and <code>ngrep</code>! And there's a whole section on <code>perf</code> because perf is the best.</p>
<p>If you don't know what any of those tools I just mentioned are -- PERFECT. You
are who this zine is for!!! Read it and find out why I love them! Also, a lot
of these tools happen to work on OS X :)</p>
<p>I've been really delighted to see that a ton of people have enjoyed & learned
something new from this zine, whether they just started using Linux (!!!) or
have been debugging on Linux for 10 years.</p>
<div align="center">
<a href="http://jvns.ca/debugging-zine.pdf"><img src="http://jvns.ca/images/debugging-tools-cover.png" /></a>
</div>
<p>As usual, there are 3 versions. If you print it, you can print as many as you
want! Give them to your friends! Teach them about tcpdump!</p>
<ul>
<li>version to read on your <a href="http://jvns.ca/debugging-zine.pdf">computer</a></li>
<li>print version: <a href="http://jvns.ca/debugging-zine-print-letter.pdf">letter</a></li>
<li>print version: <a href="http://jvns.ca/debugging-zine-print-a4.pdf">A4</a></li>
</ul>
<p>The cover art is by <a href="https://twitter.com/notwaldorf">Monica</a>, who is the best.</p>
<p><small>Thanks to my amazing partner Kamal Marhubi for endless reviews, and many many other people. It turns out that a zine project takes a long time! I do not even know how people write books.</small></p>Know Your Eels2016-07-08T19:45:13.423000ZDoughttp://www.savagechickens.com/2016/07/know-your-eels.html<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/know-your-eels/7615:761cd6">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/7615.png" style="vertical-align: middle;width:16px;height:16px;"> Savage Chickens – Cartoons on Sticky Notes by Doug Savage.</b>
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p><img alt="Know Your Eels" height="637" src="http://www.savagechickens.com/wp-content/uploads/chickeneels.jpg" width="400" /></p>
<p>The wonders of <a href="http://www.savagechickens.com/tag/biology">biology</a>.</p>Marcin Borkowski: Emacs Lisp closures demystified2016-05-17T21:47:52.695000ZMarcin Borkowskihttp://mbork.pl/2016-05-17_Emacs_Lisp_closures_demystified<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/marcin-borkowski-ema/1073:94cf9e">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/1073.png" style="vertical-align: middle;width:16px;height:16px;"> Planet Emacsen:</b>
</td>
</tr>
<tr>
<td>
It's in the details ;-)
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
It is often claimed that one of the advantages of closures (as in “lexical scoping”) is information hiding (a.k.a. encapsulation). This is true, but as this post shows, you can’t really hide anything in Emacs Lisp;-).A story about Full-Stack Javascript2016-05-09T22:14:55.832000ZCommitStriphttp://www.commitstrip.com/en/2016/05/06/a-story-about-full-stack-javascript/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/a-story-about-full-s/1600460:c79b00">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/1600460.png" style="vertical-align: middle;width:16px;height:16px;"> CommitStrip.</b>
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p><ins><img alt="" class="alignnone size-full wp-image-15533" height="919" src="http://www.commitstrip.com/wp-content/uploads/2016/05/Strip-Le-fullstack-JS-2-650-finalenglish-2.jpg" width="650"></ins></p> <p><del><img alt="Strip-Le-fullstack-JS-(2)-(650-final)(english)" class="alignnone size-full wp-image-15527" height="919" src="http://www.commitstrip.com/wp-content/uploads/2016/05/Strip-Le-fullstack-JS-2-650-finalenglish.jpg" width="650"></del></p>Laws of Physics2016-04-26T18:57:30.637000Zhttp://xkcd.com/1670/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/laws-of-physics/5994357:f401af">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/5994357.png" style="vertical-align: middle;width:16px;height:16px;"> xkcd.com.</b>
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<img alt="The laws of physics are fun to try to understand, but as an organism with incredibly delicate eyes who evolved in a world full of sharp objects, I have an awful lot of trust in biology's calibration of my flinch reflex." src="http://imgs.xkcd.com/comics/laws_of_physics.png" title="The laws of physics are fun to try to understand, but as an organism with incredibly delicate eyes who evolved in a world full of sharp objects, I have an awful lot of trust in biology's calibration of my flinch reflex." />The Very Hungry Caterpillar Who Then Came to Realize That Life Was Meaningless2016-03-19T09:39:28.884000Zhttp://existentialcomics.com/comic/124<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/the-very-hungry-cate/5275170:be9aff">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/5275170.png" style="vertical-align: middle;width:16px;height:16px;"> Existential Comics:</b>
</td>
</tr>
<tr>
<td>
Alt version
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<img src="http://static.existentialcomics.com/comics/hungryCaterpillar1.png" />
<br /><hr /><br />
<img alt="At least for a day or two, and then he realized that beauty too was empty and fleeting. But hey, at least he looked good in the meantime." src="http://static.existentialcomics.com/comics/hungryCaterpillar2.png" title="" />Three Words – Any Place2016-01-18T12:36:14.939000ZAlex Tabarrokhttp://feedproxy.google.com/~r/marginalrevolution/feed/~3/HEsg3yW7TEM/three-words-any-place.html<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/three-words-any-plac/7797882:a78506">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/7797882.png" style="vertical-align: middle;width:16px;height:16px;"> Marginal Revolution.</b>
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p></p><p>Here’s an amazing new tool. <a href="http://what3words.com/about/">what3words</a> has identified every one of the 57 trillion 3mx3m squares on the entire planet with just three, easy to remember, words. My office, for example, not my building but my office, is token.oyster.whispering. Tyler’s office just down the hall is barons.huts.sneaky. (Especially easy to remember if you recall this is Tyrone’s office as well.)</p>
<p>Every location on the earth now has a fixed, easily-accessible and memorable address. Unpopulated places have addresses for the first time ever, of course, but now so do heavily populated places like favelas in Brazil where there are no roads or numbered houses. In principle, addressing could be done with latitude and longitude but that’s like trying to direct people to web sites with IP addresses–not good for humans.</p>
<p>Algorithms have assigned words to avoid homophones (sale & sail) and to place similar combos far from one another to aid in error detection. Simpler, more common words are used to address more populated areas and longer words are used in unpopulated areas.</p>
<p>Moreover the three word addresses are available not just in English but in French, Spanish, Portuguese, Swahili, Russian, German, Turkish and Swedish with more languages on the way. The addresses in other languages are not translations but unique 3 word addresses in those languages.</p>
<p>All of this is available in a small app so that it can be used even offline on a simple smartphone. Find your address <a href="https://map.what3words.com/">here</a>.</p>
<p>Hat tip: <a href="http://thebrowser.com">The Browser</a>.</p>
<p>The post <a href="http://marginalrevolution.com/marginalrevolution/2016/01/three-words-any-place.html" rel="nofollow">Three Words – Any Place</a> appeared first on <a href="http://marginalrevolution.com" rel="nofollow">Marginal REVOLUTION</a>.</p>
<img alt="" height="1" src="http://feeds.feedburner.com/~r/marginalrevolution/feed/~4/HEsg3yW7TEM" width="1" />sachachua: Scan ~/bin and turn the scripts into Emacs commands2015-12-17T17:22:06.669000ZSacha Chuahttp://sachachua.com/blog/2015/12/scan-bin-turn-scripts-emacs-commands/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/sachachua-scan-bin-a/1073:839301">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/1073.png" style="vertical-align: middle;width:16px;height:16px;"> Planet Emacsen:</b>
</td>
</tr>
<tr>
<td>
Nice little integration for your shell scripts into Emacs.
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p>I want to automate little things on my computer so that I don’t have to look up command lines or stitch together different applications. Many of these things make sense to turn into shell scripts. That way, I can call them from other programs and assign keyboard shortcuts to them. Still, I spend most of my computer time in Emacs, and I don’t want to think about whether I’ve defined a command in Emacs Lisp or in a shell script. Besides, I like the way <a href="http://sachachua.com/blog/2014/03/emacs-basics-call-commands-name-m-x-tips-better-completion-using-ido-helm/">Helm</a> lets me type parts of commands in order to select and call them.</p>
<p>Emacs Lisp allows you to define a macro that results in Emacs Lisp code. In this case, I want to define interactive functions so I can call them with <code>M-x</code>. In case I decide to call them from Emacs Lisp, such as <code>(my/shell/rotate-screen "left")</code>, I want to be able to pass arguments. I’m also using <a href="https://github.com/magnars/dash.el">dash.el</a> to provide functions like <code>-filter</code> and <code>-not</code>, although I could rewrite this to just use the standard Emacs Lisp functions.</p>
<p>Here’s the code that scans a given directory for executable files and creates interactive functions, and some code that calls it for my <a href="https://github.com/sachac/scripts">~/bin</a> directory.</p>
<div class="org-src-container">
<pre class="src src-emacs-lisp">(<span class="org-keyword">defmacro</span> <span class="org-function-name">my/convert-shell-scripts-to-interactive-commands</span> (directory)
<span class="org-doc">"Make the shell scripts in DIRECTORY available as interactive commands."</span>
(cons 'progn
(-map
(<span class="org-keyword">lambda</span> (filename)
(<span class="org-keyword">let</span> ((function-name (intern (concat <span class="org-string">"my/shell/"</span> (file-name-nondirectory filename)))))
`(<span class="org-keyword">defun</span> ,function-name (<span class="org-type">&rest</span> args)
(<span class="org-keyword">interactive</span>)
(apply 'call-process ,filename nil nil nil args))))
(-filter (-not #'file-directory-p)
(-filter #'file-executable-p (directory-files directory t))))))
(<span class="org-keyword">my/convert-shell-scripts-to-interactive-commands</span> <span class="org-string">"~/bin"</span>)
</pre>
</div>
<p>Let’s see how that goes!</p>
<p>The post <a href="http://sachachua.com/blog/2015/12/scan-bin-turn-scripts-emacs-commands/" rel="nofollow">Scan ~/bin and turn the scripts into Emacs commands</a> appeared first on <a href="http://sachachua.com/blog" rel="nofollow">sacha chua :: living an awesome life</a>.</p>How I got better at debugging2015-11-22T20:48:49.540000Zhttp://jvns.ca/blog/2015/11/22/how-i-got-better-at-debugging/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/how-i-got-better-at-/3815987:a0796c">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/3815987.png" style="vertical-align: middle;width:16px;height:16px;"> Julia Evans:</b>
</td>
</tr>
<tr>
<td>
Good write up. It's nice to go deep diving into bug hunting every now and again ;-)
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p>I had a performance review last week where I was told, among other things, that I’m very good at debugging, especially difficult & confusing problems. I thought about this and I was like YEAH I AM. But I didn’t used to be. What happened?!</p>
<p>I sometimes hear advice to be extremely systematic and organized. I think that’s good advice and I told my partner this and he laughed because I am not the most systematic and organized person. But here are some things that I think have helped me anyway:</p>
<p><strong>Remember that the bug is happening for a logical reason</strong></p>
<p>Sometimes when I hit a bug, especially a nondeterministic and difficult to reproduce bug, it’s tempting to think “oh you know, things just happen, who knows”. But everything on a computer does in fact happen for a logical reason (however much the computer may try to convince you otherwise). Reminding myself of that helps me fix bugs. Also known as “OK JULIA IT IS NOT FAIRIES WHAT ACTUAL REASON COULD BE CAUSING THIS?”</p>
<p><strong>Be unreasonably confident in my ability to fix the bug</strong></p>
<p>I recently dealt with a performance problem in a job at work that took me 3 weeks to fix (see <a href="http://jvns.ca/blog/2015/09/10/a-millisecond-isnt-fast-and-how-we-fixed-it/">a millisecond isn’t fast</a>). If I hadn’t been able to fix it, I would have felt pretty bad and like it was a waste of 3 weeks.</p>
<p>But we were processing a relatively small number of records, and it was taking 15 hours to do it, and it was NOT REASONABLE and I knew that the job was too slow. And I figured it out, and now it’s faster and everyone is happy.</p>
<p>(since I can now often actually fix bugs I tackle, perhaps this confidence is now reasonable :D)</p>
<p><strong>Know more things</strong></p>
<p>This <a href="http://jvns.ca/blog/2015/11/21/why-you-should-understand-a-little-about-tcp/">TCP bug</a> I talked about yesterday? I wouldn’t have been able to fix that in my first job out of grad school. I just didn’t understand enough about how computer networks work, or computers (I had an awesome math & theoretical CS degree and I did not learn anything about computers there.). And I didn’t know strace.</p>
<p>There’s a service at work that sometimes takes a long time to respond because of JVM garbage collection pauses. If you don’t know that a common source of latency issues on the JVM is garbage collection pauses (or worse, if you don’t know that garbage collection pauses are even a thing that happen), then you’re going to have a really bad day trying to figure that out.</p>
<p>Understanding the structure of the system I’m trying to debug and what some of the common failure modes are has been really indispensable to me.</p>
<p><strong>Talk to other people</strong></p>
<p>I sometimes just ramble into the Slack channel at work about the problem I’m working on, which sometimes looks like</p>
<div class="bogus-wrapper"><figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class="line-number">1</span>
<span class="line-number">2</span>
<span class="line-number">3</span>
<span class="line-number">4</span>
<span class="line-number">5</span>
<span class="line-number">6</span>
<span class="line-number">7</span>
</pre></td><td class="code"><pre><code class=""><span class="line">julia: i have no idea why this bug is happening
</span><span class="line">julia: i mean I tried X and it is still happening
</span><span class="line">julia: and also W
</span><span class="line">julia: and also Z
</span><span class="line">julia: OH RIGHT I FORGOT ABOUT ABC
</span><span class="line">julia: yayy
</span><span class="line">someone else: :)</span></code></pre></td></tr></table></div></figure></div>
<p>Also sometimes if I start talking about it then someone will come and talk to me and say something helpful! It’s the best.</p>
<p>I got really stuck on that 3 week bug we talked about before and got on the phone to <a href="https://twitter.com/avibryant">Avi</a>, which was VERY USEFUL because he wrote the code that I was optimizing. So in that case I didn’t just need a rubber duck, I needed to talk to someone who knew more about the code (“oh yeah we haven’t optimized that part at all yet so it’s not a surprise that it’s slow!”).</p>
<p>I’ve gotten way better at figuring out what I don’t understand, articulating it, and asking about it.</p>
<p><strong>Use strace</strong></p>
<p>Seriously I could not fix bugs without strace.</p>
<p>More generally, being able to observe directly what a program is actually doing is incredibly valuable. I was trying to debug recently why a request I was sending to Redis was invalid. And I read the code, and asked other people, and they were like “huh that looks right”. AND THEN I REMEMBERED ABOUT TCPDUMP. (tcpdump shows you the TCP traffic coming in and out of a machine. it’s the best.)</p>
<p>So I ran tcpdump on a machine that I knew was sending (valid) requests to Redis, just looked at it as ASCII in my terminal, and then all the information was right there! And I copied the valid thing into what I was testing, and it totally worked and explained everything.</p>
<p><strong>I like it more</strong></p>
<p>I used to not really like debugging. But I started being able to solve harder bugs, and now when I find a thorny debugging problem it’s way more exciting to me than writing new code. Most of the code I write is really straightforward. A difficult bug is way more likely to teach me something I didn’t know before about how computers can break.</p>
<p>❤ debugging ❤</p>One of the coders’ hardest problems2015-10-28T08:33:07.773000ZCommitStriphttp://www.commitstrip.com/en/2015/10/27/one-of-the-coders-hardest-problems/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/one-of-the-coders-ha/1600460:8406bf">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/1600460.png" style="vertical-align: middle;width:16px;height:16px;"> CommitStrip.</b>
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p><img alt="" class="aligncenter size-full wp-image-13713" height="615" src="http://www.commitstrip.com/wp-content/uploads/2015/10/Strip-Trouver-le-nom-de-variable-english650-final.jpg" width="650" /></p>Things I Have Learned About Emscripten2015-09-23T20:17:23.431000ZMultimedia Mikehttp://multimedia.cx/eggs/things-i-have-learned-about-emscripten/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/things-i-have-learne/185771:9a8c38">shared this story</a>
from <img src="https://www.newsblur.com/rss_feeds/icon/185771" style="vertical-align: middle;width:16px;height:16px;"> Breaking Eggs And Making Omelettes:</b>
</td>
</tr>
<tr>
<td>
Nice write-up of the pain points in his Emscripten project.
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p>3 years ago, I released my <a href="http://gamemusic.multimedia.cx/">Game Music Appreciation project</a>, a website with a ludicrously uninspired title which allowed users a relatively frictionless method to experience a range of specialized music files related to old video games. However, the site required use of <a href="https://chrome.google.com/webstore/detail/salty-game-music-player/leooadmebmmjogbfhdcbfldndllfkhpg?hl=en-US">a special Chrome plugin</a>. Ever since that initial release, my #1 most requested feature has been for a pure JavaScript version of the music player.</p> <p><strong>“Impossible!”</strong> I exclaimed. “There’s no way JS could ever run fast enough to run these CPU emulators and audio synthesizers in real time, and allow for the visualization that I demand!” Well, I’m pleased to report that I have proved me wrong. I recently quietly launched a new site with what I hope is a catchier title, meant to evoke a cloud-based retro-music-as-a-service product: <strong><a href="http://cirrusretro.com/">Cirrus Retro</a></strong>. Right now, it’s basically the same as the old site, but without the wonky Chrome-specific technology.</p> <p>Along the way, I’ve learned a few things about using Emscripten that I thought might be useful to share with other people who wish to embark on a similar journey. This is geared more towards someone who has a stronger low-level background (such as C/C++) vs. high-level (like JavaScript).</p> <p><strong>General Goals</strong><br> Do you want to cross-compile an entire desktop application, one that relies on an extensive GUI toolkit? That might be difficult (though I believe there is a path for porting qt code directly with Emscripten). Your better wager might be to abstract out the core logic and processes of the program and then create a new web UI to access them.</p> <p>Do you want to compile a game that basically just paints stuff to a 2D canvas? You’re in luck! Emscripten has a porting path for <a href="https://www.libsdl.org/">SDL</a>. Make a version of your C/C++ software that targets SDL (generally not a tall order) and then compile that with Emscripten.</p> <p>Do you just want to cross-compile some functionality that lives in a library? That’s what I’ve done with the Cirrus Retro project. For this, plan to compile the library into a JS file that exports some public functions that other, higher-level, native JS (i.e., JS written by a human and not a computer) will invoke.</p> <p><strong>Memory Levels</strong><br> When porting C/C++ software to JavaScript using Emscripten, you have to think on 2 different levels. Or perhaps you need to force JavaScript into a low level C lens, especially if you want to write native JS code that will interact with Emscripten-compiled code. This often means somehow allocating chunks of memory via JS and passing them to the Emscripten-compiled functions. And you wouldn’t believe the type of gymnastics you need to execute to get native JS and Emscripten-compiled JS to cooperate.<br> <span id="more-4397"></span></p> <p><a href="http://kapadia.github.io/emscripten/2013/09/13/emscripten-pointers-and-pointers.html">“Emscripten: Pointers and Pointers” is the best (and, really, ONLY) explanation</a> I could find for understanding the basic mechanics of this process, at least when I started this journey. However, there’s a mistake in the explanation that left me confused for a little while, and I’m at a loss to contact the author (doesn’t anyone post a simple email address anymore?).</p> <p>Per the best of my understanding, Emscripten allocates a large JS array and calls that the memory space that the compiled C/C++ code is allowed to operate in. A pointer in C/C++ code will just be an index into that mighty array. Really, that’s not too far off from how a low-level program process is supposed to view memory– as a flat array.</p> <p>Eventually, I just learned to cargo-cult my way through the memory allocation process. Here’s the JS code for allocating an Emscripten-compatible byte buffer, taken from my test harness (more on that later):</p> <pre>var musicBuffer = fs.readFileSync(testSpec['filename']);
var musicBufferBytes = new Uint8Array(musicBuffer);
var bytesMalloc = player._malloc(musicBufferBytes.length);
var bytes = new Uint8Array(player.HEAPU8.buffer, bytesMalloc, musicBufferBytes.length);
bytes.set(new Uint8Array(musicBufferBytes.buffer));
</pre> <p>So, read the array of bytes from some input source, create a Uint8Array from the bytes, use the Emscripten _malloc() function to allocate enough bytes from the Emscripten memory array for the input bytes, then create a new array… then copy the bytes… </p> <p>You know what? It’s late and I can’t remember how it works exactly, but it does. It has been a few months since I touched that code (been fighting with front-end website tech since then). You write that memory allocation code enough times and it begins to make sense, and then you hope you don’t have to write it too many more times.</p> <p><strong>Multithreading</strong><br> <strong>You can’t port multithreaded code to JS via Emscripten. JavaScript has no notion of threads!</strong> If you don’t understand the computer science behind this limitation, a more thorough explanation is beyond the scope of this post. But trust me, I’ve thought about it a lot. In fact, <a href="https://kripken.github.io/emscripten-site/docs/porting/guidelines/portability_guidelines.html">the official Emscripten literature states</a> that you should be able to port most any C/C++ code as long as 1) none of the code is proprietary (i.e., all the raw source is available); and 2) there are no threads.</p> <p>Yes, I read about <a href="https://groups.google.com/forum/#!topic/emscripten-discuss/gQQRjajQ6iY">the experimental pthreads support added to Emscripten recently</a>. Don’t get too excited; that won’t be ready and widespread for a long time to come as it relies on a new browser API. In the meantime, figure out how to make your multithreaded C/C++ code run in a single thread if you want it to run in a browser.</p> <p><strong>Printing Facility</strong><br> Eventually, getting software to work boils down to debugging, and the most primitive tool in many a programmer’s toolbox is the humble print statement. A print statement allows you to inspect a piece of a program’s state at key junctures. Eventually, when you try to cross-compile C/C++ code to JS using Emscripten, something is not going to work correctly in the generated JS “object code” and you need to understand what. You’ll be pleading for a method of just inspecting one variable deep in the original C/C++ code.</p> <p>I came up with this simple printf-workalike called emprintf():</p> <pre>#ifndef EMPRINTF_H
#define EMPRINTF_H
#include <stdio .h>
#include <stdarg .h>
#include <emscripten .h>
#define MAX_MSG_LEN 1000
/* NOTE: Don't pass format strings that contain single quote (') or newline
* characters. */
static void emprintf(const char *format, ...)
{
char msg[MAX_MSG_LEN];
char consoleMsg[MAX_MSG_LEN + 16];
va_list args;
/* create the string */
va_start(args, format);
vsnprintf(msg, MAX_MSG_LEN, format, args);
va_end(args);
/* wrap the string in a console.log('') statement */
snprintf(consoleMsg, MAX_MSG_LEN + 16, "console.log('%s')", msg);
/* send the final string to the JavaScript console */
emscripten_run_script(consoleMsg);
}
#endif /* EMPRINTF_H */
</pre> <p>Put it in a file called “emprint.h”. Include it into any C/C++ file where you need debugging visibility, use emprintf() as a replacement for printf() and the output will magically show up on the browser’s JavaScript debug console. Heed the comments and don’t put any single quotes or newlines in strings, and keep it under 1000 characters. I didn’t say it was perfect, but it has helped me a lot in my Emscripten adventures.</p> <p><strong>Optimization Levels</strong><br> Remember to turn on optimization when compiling. <a href="http://multimedia.cx/eggs/playing-with-emscripten-and-asm-js/">I have empirically found that optimizing for size (-Os) leads to the best performance</a> all around, in addition to having the smallest size. Just be sure to specify <em>some</em> optimization level. If you don’t, the default is -O0 which offers horrible performance when running in JS.</p> <p><strong>Static Compression For HTTP Delivery</strong><br> JavaScript code compresses pretty efficiently, even after it has been optimized for size using -Os. I routinely see compression ratios between 3.5:1 and 5:1 using gzip.</p> <p>Web servers in this day and age are supposed to be smart enough to detect when a requesting web browser can accept gzip-compressed data and do the compression on the fly. They’re even supposed to be smart enough to cache compressed output so the same content is not recompressed for each request. I would have to set up a series of tests to establish whether either of the foregoing assertions are correct and I can’t be bothered. Instead, I took it into my own hands. The trick is to pre-compress the JS files and then instruct the webserver to serve these files with a ‘Content-Type’ of ‘application/javascript’ and a ‘Content-Encoding’ of ‘gzip’.</p> <ol><li>Compress your large Emscripten-build JS files with ‘gzip’: ‘gzip compiled-code.js’</li> <li>Rename them from extension .js.gz to .jsgz</li> <li>Tell the webserver to deliver .jsgz files with the correct Content-Type and Content-Encoding headers</li> </ol> <p>To do that last step with Apache, specify these lines:</p> <pre>AddType application/javascript jsgz
AddEncoding gzip jsgz
</pre> <p>They belong in either a directory’s .htaccess file or in the sitewide configuration (/etc/apache2/mods-available/mime.conf works on my setup).</p> <p><strong>Build System and Build Time Optimization</strong><br> Oh goodie, build systems! I had a very specific manner in which I wanted to build my JS modules using Emscripten. Can I possibly coerce any of the many popular build systems to do this? It has been a few months since I worked on this problem specifically but I seem to recall that the build systems I tried to used would freak out at the prospect of compiling stuff to a final binary target of .js.</p> <p>I had high hopes for <ins><a href="http://bazel.io/">Bazel</a>,</ins> <del>Bazel,</del> which Google released while I was developing Cirrus Retro. Surely, this is software that has been battle-tested in the harshest conditions of one of the most prominent software-developing companies in the world, needing to take into account the most bizarre corner cases and still build efficiently and correctly every time. And I have little doubt that it fulfills the order. Similarly, I’m confident that Google also has a team of no fewer than 100 or so people dedicated to developing and supporting the project within the organization. When you only have, at best, 1-2 hours per night to work on projects like this, you prefer not to fight with such cutting edge technology and after losing 2 or 3 nights trying to make a go of Bazel, I eventually put it aside.</p> <p>I also tried to use Autotools. <a href="https://plus.google.com/115891738491519168693/posts/TPRuaVjxv2q">It failed horribly for me</a>, mostly for my own carelessness and lack of early-project source control.</p> <p>After that, it was strictly vanilla makefiles with no real dependency management. But you know what helps in these cases? <strong><a href="https://ccache.samba.org">ccache!</a></strong> Or at least, it would if it didn’t fail with Emscripten.</p> <p>Quick tip: ccache has trouble with LLVM unless you set the <strong>CCACHE_CPP2</strong> environment variable (e.g.: “export CCACHE_CPP=1”). I don’t remember the specifics, but it magically fixes things. Then, the lazy build process becomes “make clean && make”.</p> <p><strong>Testing</strong><br> If you have never used Node.js, testing Emscripten-compiled JS code might be a good opportunity to start. I was able to use Node.js to great effect for testing the individually-compiled music player modules, wiring up a series of invocations using Python for a broader test suite (wouldn’t want to go too deep down the JS rabbit hole, after all).</p> <p>Be advised that Node.js doesn’t enjoy the same kind of JIT optimizations that the browser engines leverage. Thus, in the case of time critical code like, say, an audio synthesis library, the code might not run in real time. But as long as it produces the correct bitwise waveform, that’s good enough for continuous integration.</p> <p>Also, if you have largely been a low-level programmer for your whole career and are generally unfamiliar with the world of single-threaded, event-driven, callback-oriented programming, you might be in for a bit of a shock. When I wanted to learn how to read the contents of a file in Node.js, <a href="http://www.sitepoint.com/accessing-the-file-system-in-node-js/">this is the first tutorial I found on the matter</a>. I thought the code presented was a parody of bad coding style:</p> <pre>var fs = require("fs");
var fileName = "foo.txt";
fs.exists(fileName, function(exists) {
if (exists) {
fs.stat(fileName, function(error, stats) {
fs.open(fileName, "r", function(error, fd) {
var buffer = new Buffer(stats.size);
fs.read(fd, buffer, 0, buffer.length, null, function(error, bytesRead, buffer) {
var data = buffer.toString("utf8", 0, buffer.length);
console.log(data);
fs.close(fd);
});
});
});
}});
</pre> <p>Apparently, this kind of thing doesn’t raise an eyebrow in the JS world.</p> <p>Now, I understand and respect the JS programming model. But this was seriously frustrating when I first encountered it because a simple script like the one I was trying to write just has an ordered list of tasks to complete. When it asks for bytes from a file, it really has nothing better to do than to wait for the answer.</p> <p>Thankfully, it turns out that <a href="https://nodejs.org/api/fs.html">Node’s fs module</a> includes synchronous versions of the various file access functions. So it’s all good.</p> <p><strong>Conclusion</strong><br> I’m sure I missed or underexplained some things. But if other brave souls are interested in dipping their toes in the waters of Emscripten, I hope these tips will come in handy.</p>Irreal: Emacs and Data Science2015-07-06T12:36:36.939000Zjcshttp://irreal.org/blog/?p=4173<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/irreal-emacs-and-dat/815877:0df37c">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/815877.png" style="vertical-align: middle;width:16px;height:16px;"> A Better Planet Emacs:</b>
</td>
</tr>
<tr>
<td>
I'm not a data scientist but I do use org-mode a lot to keep my log book along with working samples of code.
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p>Robert Vesco has an interesting post on <a href="http://www.insightdatascience.com/blog/emacs_for_data_science.html" rel="nofollow" target="_blank">why he uses Emacs in his data science work</a>. Vesco is a data scientist for Bloomberg so he's a serious practitioner of the art. Working in data science means he uses a variety of languages such as Python, R, SQL, Stata, and SAS. He notes that most of those languages have an associated IDE that simplifies working with them but that that means learning multiple editors and probably mastering none. He also notes that those specialized IDEs may fall out of favor and not be supported in the future, that they are not portable across platforms, and that they are hard to customize. </p>
<p> Happily, those defects do not apply to Emacs. It runs on essentially every (serious) platform, is open source, will be supported for as long as there are a few programmers still interested in using it, and, of course, is famously customizable. One consequence of that customizability is it can become a reasonable IDE for almost any language. That means that a single tool can be used for all those languages and that it's worthwhile mastering that tool because it's the only (editing) tool you need to learn. </p>
<p> The bulk of Vesco's post covers those features of Emacs that he finds most useful in his work. One of those features is Org mode that allows him to use reproducible research methods in his research and publishing. It's an interesting read even if you're not a data scientist. </p>Team Effort2015-07-04T12:10:05.299000Zhttp://xkcd.com/1543/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/team-effort/5994357:57ac2f">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/5994357.png" style="vertical-align: middle;width:16px;height:16px;"> xkcd.com.</b>
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<img alt="Given the role they play in every process in my body, really, they deserve this award more than me. Just gotta figure out how to give it to them. Maybe I can cut it into pieces to make it easier to swallow ..." src="http://imgs.xkcd.com/comics/team_effort.png" title="Given the role they play in every process in my body, really, they deserve this award more than me. Just gotta figure out how to give it to them. Maybe I can cut it into pieces to make it easier to swallow ..." />Chris Ball: Announcing GitTorrent: A Decentralized GitHub2015-06-03T09:15:33.825000Zcjbhttp://blog.printf.net/articles/2015/05/29/announcing-gittorrent-a-decentralized-github/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/chris-ball-announcin/1073:ed535a">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/1073.png" style="vertical-align: middle;width:16px;height:16px;"> Planet Emacsen:</b>
</td>
</tr>
<tr>
<td>
Does this mean programs like aacskeys and DumpHD can have their code hosted in a source repository that can't get DMCA'd?
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p>(This post is an <i>aspirational transcript</i> of the talk I gave to the <a href="http://dtn.is">Data Terra Nemo</a> conference in May 2015. If you’d like to watch the less eloquent version of the same talk that I <i>actually</i> gave, the video should be available soon!)</p>
<p>I’ve been working on building a decentralized GitHub, and I’d like to talk about what this means and why it matters — and more importantly, show you how it can be done and real GitTorrent code I’ve implemented so far.</p>
<h4>Why a decentralized GitHub?</h4>
<p>First, the practical reasons: GitHub might become untrustworthy, get hacked — or get DDOS’d by China, as <a href="http://arstechnica.com/security/2015/03/massive-denial-of-service-attack-on-github-tied-to-chinese-government/">happened</a> while I was working on this project! I know GitHub seems to be doing many things right at the moment, but there often comes a point at which companies that have raised $100M in Venture Capital funding start making decisions that their users would strongly prefer them not to.</p>
<p>There are philosophical reasons, too: GitHub is closed source, so we can’t make it better ourselves. <a href="http://mako.cc">Mako Hill</a> has an essay called <a href="http://mako.cc/writing/hill-free_tools.html">Free Software Needs Free Tools</a>, which describes the problems with depending on proprietary software to produce free software, and I think he’s right. To look at it another way: the experience of our collaboration around open source projects is currently being defined by the unmodifiable tools that GitHub has decided that we should use.</p>
<p>So that’s the practical and philosophical, and I guess I’ll call the third reason the “ironical”. It is a massive irony to move from many servers running the CVS and Subversion protocols, to a single centralized server speaking the decentralized Git protocol. Google Code <a href="http://google-opensource.blogspot.com/2015/03/farewell-to-google-code.html">announced its shutdown</a> a few months ago, and their rationale was explicitly along the lines of “everyone’s using GitHub anyway, so we don’t need to exist anymore”. We’re quickly heading towards a single central service for all of the world’s source code.</p>
<p>So, especially at this conference, I expect you’ll agree with me that this level of centralization is unwise.</p>
<h4>Isn’t Git already decentralized?</h4>
<p>You might be thinking that while GitHub is centralized, the Git protocol is decentralized — when you clone a repository, your copy is as good as anyone else’s. Isn’t that enough?</p>
<p>I don’t think so, and to explain why I’d like you to imagine someone arguing that we can do without BitTorrent because we have FTP. We would not advocate replacing BitTorrent with FTP, and the suggestion doesn’t even make sense! First — there’s no index of which hosts have which files in FTP, so we wouldn’t know where to look for anything. And second — even if we knew who owned copies of the file we wanted, those computers aren’t going to be running an anonymous FTP server.</p>
<p>Just like Git, FTP doesn’t turn clients into servers in the way that a peer-to-peer protocol does. So that’s why Git isn’t <i>already</i> the decentralized GitHub — you don’t know where anything’s stored, and even if you did, those machines aren’t running Git servers that you’re allowed to talk to. I think we can fix that.</p>
<h4>Let’s GitTorrent a repo!</h4>
<p>Let’s jump in with a demo of GitTorrent – that is, cloning a Git repository that’s hosted on BitTorrent:</p>
<pre><code class="block"><span class="nf">1</span> <span class="bolder">λ</span> git clone gittorrent://github.com/cjb/recursers
<span class="nf">2</span> Cloning into 'recursers'...
<span class="nf">3</span>
<span class="nf">4</span> Okay, we want to get: <span class="ss">5fbfea8de70ddc686dafdd24b690893f98eb9475</span>
<span class="nf">5</span>
<span class="nf">6</span> Adding swarm peer: <span class="ss">192.34.86.36:30000</span>
<span class="nf">7</span>
<span class="nf">8</span> Downloading git pack with infohash: <span class="ss">9d98510a9fee5d3f603e08dcb565f0675bd4b6a2</span>
<span class="nf">9</span>
<span class="nf">10</span> Receiving objects: 100% (47/47), 11.47 KiB | 0 bytes/s, done.
<span class="nf">11</span> Resolving deltas: 100% (10/10), done.
<span class="nf">12</span> Checking connectivity... done.</code></pre>
<p>Hey everyone: we just cloned a git repository over BitTorrent! So, let’s go through this line by line.</p>
<p><b>Lines 1-2:</b> Git actually has an extensible mechanism for network protocols built in. The way it works is that my <code>git clone</code> line gets turned into “run the <code>git-remote-gittorrent</code> command and give it the URL as an argument”. So we can do whatever we want to perform the actual download, and we’re responsible for writing git objects into the new directory and telling Git when we’re done, and we didn’t have to modify Git at all to make this work.</p>
<p>So <code>git-remote-gittorrent</code> takes it from here. First we connect to GitHub to find out what the latest revision for this repository is, so that we know what we want to get. GitHub tells us it’s <code>5fbfea8de..</code>.</p>
<p><b>Lines 4-6:</b> Then we go out to the GitTorrent network, which is a distributed hash table just like BitTorrent’s, and ask if anyone has a copy of commit <code>5fbdea8de..</code>. Someone said yes! We make a BitTorrent connection to them. The way that BitTorrent’s distributed hash table works is that there’s a single operation, <code>get_nodes(hash)</code> which tells you who can send you content that you want, like this:</p>
<pre><code class="block">get_nodes(<span class="s">'5fbfea8de70ddc686dafdd24b690893f98eb9475'</span>) =
[<span class="nf">192.34.86.36</span>:<span class="nf">30000</span>, ...]
</code></pre>
<p>Now, in standard BitTorrent with “trackerless torrents”, you ask for the <i>files</i> that you want by their content, and you’d get them and be happy. But a repository the size of the Linux kernel has four million commits, so just receiving the one commit <code>5fbdea8de..</code> wouldn’t be helpful; we’d have to make another four million requests for all the other commits too. Nor do we want to get every commit in the repository every time we ‘git pull’. So we have to do something else.</p>
<p><b>Lines 8-12:</b> Git has solved this problem — it has this “smart protocol format” for negotiating an exchange of git objects. We can think of it this way:</p>
<p>Imagine that your repository has 20 commits, 1-20. And the 15th commit is <code>bbbb</code> and the most recent 20th commit is <code>aaaa</code>. The Git protocol negotiation would look like this:</p>
<pre><code class="block">1> have aaaa
2> want aaaa
2> have bbbb
</code></pre>
<p>Because of the way the git graph works, node 1> here can look up where <code>bbbb</code> is on the graph, see that you’re only asking for five commits, and create you a “packfile” with just those objects. Just by a three-step communication.</p>
<p>That’s what we’re doing here with GitTorrent. We ask for the commit we want and connect to a node with BitTorrent, but once connected we conduct this Smart Protocol negotiation in an overlay connection on top of the BitTorrent wire protocol, in what’s called a BitTorrent Extension. Then the remote node makes us a packfile and tells us the hash of that packfile, and then we start downloading that packfile from it and any other nodes who are seeding it using Standard BitTorrent. We can authenticate the packfile we receive, because after we uncompress it we know which Git commit our graph is supposed to end up at; if we don’t end up there, the other node lied to us, and we should try talking to someone else instead.</p>
<p>So that’s what just happened in this terminal. We got a packfile made for us with this hash — and it’s one that includes every object because this is a fresh clone — we downloaded and unpacked it, and now we have a local git repository.</p>
<p>This was a git clone where everything up to the actual downloading of git objects happened as it would in the normal GitHub way. If GitHub decided tomorrow that it’s sick of being in the disks and bandwidth business, it could encourage its users to run this version of GitTorrent, and it would be like having a peer to peer <a href="http://en.wikipedia.org/wiki/Content_delivery_network">“content delivery network”</a> for GitHub, falling back to using GitHub’s servers in the case where the commits you want aren’t already present in the CDN.</p>
<h4>Was that actually decentralized?</h4>
<p>That’s some progress, but you’ll have noticed that the very first thing we did was talk to GitHub to find out which hash we were ultimately aiming for. If we’re really trying to decentralize GitHub, we’ll need to do much better than that, which means we need some way for the owner of a repository to let us know what the hash of the latest version of that repository is. In short, we now have a global database of git objects that we can download, but now we need to know what objects we want — we need to emulate the part of github where you go to <code>/user/repo</code>, and you know that you’re receiving the very latest version of that user’s repo.</p>
<p>So, let’s do better. When all you have is a hammer, everything looks like a nail, and my hammer is this distributed hash table we just built to keep track of which nodes have which commits. Very recently, <a href="http://substack.net/">substack</a> noticed that there’s a BitTorrent extension for making each node be partly responsible for maintaining a network-wide key-value store, and he coded it up. It adds two more operations to the DHT, <code>get()</code> and <code>put()</code>, and <code>put()</code> gives you 1000 bytes per key to place a message into the network that can be looked up later, with your answer repeated by other nodes after you’ve left the network. There are two types of key — the first is immutable keys, which work as you might expect, you just take the hash of the data you want to store, and your data is stored with that hash as the key.</p>
<p>The second type of key is a mutable key, and in this case the key you look up is the hash of a public key to a crypto keypair, and the owner of that keypair can publish signed updates as values under that key. Updates come with a sequence number, so anytime a client sees an update for a mutable key, it checks if the update has a newer sequence number than the value it’s currently recorded, and it checks if the update is signed by the public key corresponding to the hash table key, which proves that the update came from the key’s owner. If both of those things are true then it’ll update to this newer value and start redistributing it. This has many possible uses, but my use for it is as the place to store what your repositories are called and what their latest revision is. So you’d make a local Git commit, push it to the network, and push an update to your personal mutable key that reflects that there’s a new latest commit. Here’s a code description of the new operations:</p>
<pre><code class="block"><span class="nf">// Immutable key put</span>
hash(value) = put({
value: <span class="s">'some data'</span>
})
<span class="nf">// Mutable key put</span>
hash(key) = put({
value: <span class="s">'some data'</span>,
key: key,
seq: n
})
<span class="nf">// Get</span>
value = get(hash)</code></pre>
<p>So now if I want to tell someone to clone my GitHub repo on GitTorrent, I don’t give them the github.com URL, instead I give them this long hex number that is the hash of my public key, which is used as a mutable key on the distributed hash table.</p>
<p>Here’s a demo of that:</p>
<pre><code class="block"><span class="bolder">λ</span> git clone gittorrent://81e24205d4bac8496d3e13282c90ead5045f09ea/recursers
Cloning into 'recursers'...
Mutable key <span class="ss">81e24205d4bac8496d3e13282c90ead5045f09ea</span> returned:
<span class="nf">name</span>: Chris Ball
<span class="nf">email</span>: chris@printf.net
<span class="nf">repositories</span>:
<span class="nf">recursers</span>:
<span class="nf">master</span>: 5fbfea8de70ddc686dafdd24b690893f98eb9475
Okay, we want to get: <span class="ss">5fbfea8de70ddc686dafdd24b690893f98eb9475</span>
Adding swarm peer: <span class="ss">192.34.86.36:30000</span>
Downloading git pack with infohash: <span class="ss">9d98510a9fee5d3f603e08dcb565f0675bd4b6a2</span>
Receiving objects: 100% (47/47), 11.47 KiB | 0 bytes/s, done.
Resolving deltas: 100% (10/10), done.
Checking connectivity... done.</code></pre>
<p>In this demo we again cloned a Git repository over BitTorrent, but we didn’t need to talk to GitHub at all, because we found out what commit we were aiming for by asking our distributed hash table instead. Now we’ve got true decentralization for our Git downloads!</p>
<p>There’s one final dissatisfaction here, which is that long strings of hex digits do not make convenient usernames. We’ve actually reached the limits of what we can achieve with our trusty distributed hash table, because usernames are rivalrous, meaning that two different people could submit updates claiming ownership of the same username, and we wouldn’t have any way to resolve their argument. We need a method of “distributed consensus” to give out usernames and know who their owners are. The method I find most promising is actually Bitcoin’s <a href="http://en.wikipedia.org/wiki/Block_chain_(transaction_database)">blockchain</a> — the shared consensus that makes this cryptocurrency possible.</p>
<p>The deal is that there’s a certain type of Bitcoin transaction, called an <a href="http://blog.coinprism.com/2015/02/11/80-bytes-op-return/">OP_RETURN transaction</a>, that instead of transferring money from one wallet to another, leaves a comment as your transaction that gets embedded in the blockchain forever. Until recently you were limited to 40 bytes of comment per transaction, and it’s been <a href="https://github.com/bitcoin/bitcoin/commit/fcf646c9b08">raised to 80 bytes per transaction</a> as of Bitcoin Core 0.11. Making any Bitcoin transaction on the blockchain I believe currently costs around $0.08 USD, so you pay your 8 cents to the miners and the network in compensation for polluting the blockchain with your 80 bytes of data.</p>
<p>If we can leave comments on the blockchain, then we can leave a comment saying “Hey, I’d like the username Chris, and the hash of my public key is <i><x></i>“, and if multiple people ask for the same username, this time we’ll all agree on which public key asked for it first, because blockchains are an append-only data structure where everyone can see the full history. That’s the real beauty of Bitcoin — this currency stuff is frankly kind of uninteresting to me, but they figured out how to solve distributed consensus in a robust way. So the comment in the transaction might be:</p>
<pre><code class="block">@gittorrent!cjb!81e24205d4bac8496d3e13282c90ead5045f09ea
(<span class="nf">@service</span>!<span class="ss">username</span>!<span class="s">pubkey</span>)</code></pre>
<p>It’s interesting, though — maybe that “gittorrent” at the beginning doesn’t have to be there at all. Maybe this could be a way to register one username for every site that’s interested in decentralized user accounts with Bitcoin, and then you’d already own that username on all of them. This could be a separate module, a separate software project, that you drop in to your decentralized app to get user accounts that Just Work, in Python or Node or Go or whatever you’re writing software in. Maybe the app would monitor the blockchain and write to a database table, and then there’d be a plugin for web and network service frameworks that knows how to understand the contents of that table.</p>
<p>It surprised me that nothing like this seems to exist already in the decentralization community. I’d be happy to work on a project like this and make GitTorrent sit on top of it, so please let me know if you’re interested in helping with that.</p>
<p>By the way, username registration becomes a little more complicated than I just said, because the miners could see your message, and decide to replace it before adding it to the blockchain, as a registration of your username to <i>them</i> instead of you. This is the equivalent of going to a domain name registrar and typing the domain you want in their search box to see if it’s available — and at that moment of your search the registrar could turn around and register it for themselves, and then tell you to pay them a thousand bucks to give it to you. It’s no good.</p>
<p>If you care about avoiding this, Bitcoin has a way around it, and it works by making registration a two-step process. Your first message would be asking to reserve a username by supplying just the hash of that username. The miners don’t know from the hash what the username is so they can’t beat you to registering it, and once you see that your reservation’s been included in the blockchain and that no-one else got a reservation in first, you can send on a second comment that says “okay, now I want to use my reservation token, and here’s the plain text of that username that I reserved”. Then it’s yours. </p>
<p>(I didn’t invent this scheme. There’s a project called <a href="https://github.com/telehash/blockname">Blockname</a>, from <a href="https://twitter.com/jeremie">Jeremie Miller</a>, that works in exactly this way, using Bitcoin’s OP_RETURN transaction for DNS registrations on bitcoin’s blockchain. The only difference is that Blockname is performing domain name registrations, and I’m performing a mapping from usernames to hashes of public keys. I’ve also just been pointed at <a href="https://github.com/namesystem/blockstore">Blockstore</a>, which is extremely similar.)</p>
<p>So to wrap up, we’ve created a global BitTorrent swarm of Git objects, and worked on user account registration so that we can go from a user experience that looks like this:</p>
<pre><code class="block">git clone gittorrent://github.com/cjb/foo</code></pre>
<p>to this:</p>
<pre><code class="block">git clone gittorrent://81e24205d4bac8496d3e13282c90ead5045f09ea/foo</code></pre>
<p>to this:</p>
<pre><code class="block">git clone gittorrent://cjb/foo</code></pre>
<p>And at this point I think we’ve arrived at a decentralized replacement for the core feature of GitHub: finding and downloading Git repositories.</p>
<h4>Closing thoughts</h4>
<p>There’s still plenty more to do — for example, this doesn’t do anything with comments or issues or pull requests, which are all very important aspects of GitHub. </p>
<p>For issues, the solution I like is actually storing issues in files inside the code repository, which gives you nice properties like merging a branch means applying both the code changes and the issue changes — such as resolving an issue — on that branch. One implementation of this idea is <a href="http://bugseverywhere.org">Bugs Everywhere</a>.</p>
<p>We could also imagine issues and pull requests living on <a href="https://github.com/ssbc/secure-scuttlebutt">Secure Scuttlebutt</a>, which synchronizes append-only message streams across decentralized networks.</p>
<p>I’m happy just to have got this far, though, and I’d love to hear your comments on this design. The design of GitTorrent itself is (ironically enough) <a href="https://github.com/cjb/GitTorrent/blob/master/README.md">on GitHub</a> and I’d welcome pull requests to make any aspect of it better.</p>
<p>I’d like to say a few thank yous — first to <a href="http://feross.org">Feross Aboukhadijeh</a>, who wrote the BitTorrent libraries that I’m using here. Feross’s enthusiasm for peer-to-peer and the way that he runs community around his “mad science” projects made me feel excited and welcome to contribute, and that’s part of why I ended up working on this project.</p>
<p>I’m also able to work on this because I’m taking time off from work at the moment to attend the <a href="https://www.recurse.com">Recurse Center</a> in New York City. This is the place that used to be called “Hacker School” and it <a href="https://www.recurse.com/blog/77-hacker-school-is-now-the-recurse-center">changed its name</a> recently; the first reason for the name change was that they wanted to get away from the connotations of a school where people are taught things, when it’s really more like a retreat for programmers to improve their programming through project work for three months, and I’m very thankful to them for allowing me to attend.</p>
<p>The second reason they decided to change their name because their international attendees kept showing up at the US border and saying “I’m here for Hacker School!” and.. they didn’t have a good time.</p>
<p>Finally, I’d like to end with a few more words about why I think this type of work is interesting and important. There’s a certain grand, global scale of project, let’s pick GitHub and Wikipedia as exemplars, where the only way to have the project be able to exist at global scale after it becomes popular is to raise tens of millions of dollars a year, as GitHub and Wikipedia have, to spend running it, hoarding disks and bandwidth in big data centers. That limits the kind of projects we can create and imagine at that scale to those that we can make a business plan for raising tens of millions of dollars a year to run. I hope that having decentralized and peer to peer algorithms allows us to think about creating ambitious software that doesn’t require that level of investment, and just instead requires its users to cooperate and share with each other.</p>
<p>Thank you all very much for listening.</p>
<p><i>(You can check out <a href="http://github.com/cjb/gittorrent">GitTorrent on GitHub</a>, and discuss it <a href="https://news.ycombinator.com/item?id=9625840">on Hacker News</a>. You could also <a href="https://twitter.com/cjbprime">follow me on Twitter</a>.)</i></p>Emscripten and Web Audio API2015-04-29T13:44:13.482000ZMultimedia Mikehttp://multimedia.cx/eggs/emscripten-and-web-audio-api/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/emscripten-and-web-a/185771:a5ec64">shared this story</a>
from <img src="https://www.newsblur.com/rss_feeds/icon/185771" style="vertical-align: middle;width:16px;height:16px;"> Breaking Eggs And Making Omelettes:</b>
</td>
</tr>
<tr>
<td>
Another cool emscripten story.
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p><strong>Ha! They said it couldn’t be done!</strong> Well, to be fair, <a href="http://multimedia.cx/eggs/playing-with-emscripten-and-asm-js/"><strong>I said</strong> it couldn’t be done</a>. Or maybe that I just didn’t have any plans to do it. But I did it– I used <a href="http://kripken.github.io/emscripten-site/">Emscripten</a> to cross-compile a CPU-intensive C/C++ codebase (<a href="http://www.slack.net/~ant/libs/audio.html">Game Music Emu</a>) to JavaScript. Then I leveraged <a href="https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API">the Web Audio API</a> to output audio and visualize the audio using an HTML5 canvas.</p>
<p><a href="http://gamemusic.multimedia.cx/js-prototype/"><strong>Want to see it in action?</strong> Here’s a demonstration.</a> Perhaps I will be able to expand the reach of my <a href="http://gamemusic.multimedia.cx/">Game Music site</a> when I can drop the odd Native Client plugin. This JS-based player works great on Chrome, Firefox, and Safari across desktop operating systems.</p>
<p><em>But this endeavor was not without its challenges.</em></p>
<p><strong>Programmatically Generating Audio</strong><br />
First, I needed to figure out the proper method for procedurally generating audio and making it available to output. Generally, there are 2 approaches for audio output:</p>
<ol>
<li>Sit in a loop and generate audio, writing it out via a blocking audio call</li>
<li>Implement a callback that the audio system can invoke in order to generate more audio when needed</li>
</ol>
<p>Option #1 is not a good idea for an event-driven language like JavaScript. So I hunted through the rather flexible Web Audio API for a method that allowed something like approach #2. Callbacks are everywhere, after all.</p>
<p>I eventually found what I was looking for with the <a href="https://developer.mozilla.org/en-US/docs/Web/API/ScriptProcessorNode">ScriptProcessorNode</a>. It seems to be intended to apply post-processing effects to audio streams. A program registers a callback which is passed configurable chunks of audio for processing. I subverted this by simply overwriting the input buffers with the audio generated by the Emscripten-compiled library.</p>
<p>The ScriptProcessorNode interface is fairly well documented and works across multiple browsers. <strong>However</strong>, it is already marked as <strong>deprecated</strong>:</p>
<blockquote><p>Note: As of the August 29 2014 Web Audio API spec publication, this feature has been marked as deprecated, and is soon to be replaced by Audio Workers.</p></blockquote>
<p>Despite being marked as deprecated for 8 months as of this writing, there exists no appreciable amount of documentation for the successor API, these so-called Audio Workers.</p>
<p><em>Vive la web standards!</em></p>
<p><strong>Visualize This</strong><br />
The next problem was visualization. The Web Audio API provides the <a href="https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Visualizations_with_Web_Audio_API">AnalyzerNode</a> API for accessing both time and frequency domain data from a running audio stream (and fetching the data as both unsigned bytes or floating-point numbers, depending on what the application needs). This is a pretty neat idea. I just wish I could make the API work. <a href="http://chimera.labs.oreilly.com/books/1234000001552/ch05.html">The simple demos I could find</a> worked well enough. But when I wired up a prototype to fetch and visualize the time-domain wave, all I got were center-point samples (an array of values that were all 128).</p>
<p>Even if the API did work, I’m not sure if it would have been that useful. Per my reading of the AnalyserNode API, it only returns data as a single channel. Why would I want that? My application supports audio with 2 channels. <strong>I want 2 channels of data</strong> for visualization.</p>
<p><strong>How To Synchronize</strong><br />
So I rolled my own visualization solution by maintaining a circular buffer of audio when samples were being generated. Then, requestAnimationFrame() provided the rendering callbacks. The next problem was audio-visual sync. But that certainly is not unique to this situation– maintaining proper A/V sync is a perennial puzzle in real-time multimedia programming. I was able to glean enough timing information from the environment to achieve reasonable A/V sync (verify for yourself).</p>
<p><strong>Pause/Resume</strong><br />
The next problem I encountered with the Web Audio API was pause/resume facilities, or the lack thereof. For all its bells and whistles, the API’s omission of such facilities seems most unusual, as if the design philosophy was, “Once the user starts playing audio, they will never, ever have cause to pause the audio.”</p>
<p>Then again, I must understand that mine is not a use case that the design committee considered and I’m subverting the API in ways the designers didn’t intend. Typical use cases for this API seem to include such workloads as:</p>
<ul>
<li>Downloading, decoding, and playing back a compressed audio stream via the network, applying effects, and visualizing the result</li>
<li>Accessing microphone input, applying effects, visualizing, encoding and sending the data across the network</li>
<li>Firing sound effects in a gaming application</li>
<li><a href="http://mudcu.be/midi-js/">MIDI playback via JavaScript</a> (this honestly amazes me)</li>
</ul>
<p>What they did not seem to have in mind was what I am trying to do– synthesize audio in real time.</p>
<p>I implemented pause/resume in a sub-par manner: pausing has the effect of generating 0 values when the ScriptProcessorNode callback is invoked, while also canceling any animation callbacks. Thus, audio output is technically still occurring, it’s just that the audio is pure silence. It’s not a great solution because CPU is still being used.</p>
<p><strong>Future Work</strong><br />
I have a lot more player libraries to port to this new system. But I think I have a good framework set up.</p>The Biggest Little SF Publisher you never heard of pulls on the jackboots2015-04-09T06:34:24.543000ZCharlie Strosshttp://www.antipope.org/charlie/blog-static/2015/04/the-biggest-little-sf-publishe.html<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/the-biggest-little-s/3064:7c4f55">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/3064.png" style="vertical-align: middle;width:16px;height:16px;"> Charlie's Diary:</b>
</td>
</tr>
<tr>
<td>
Is this what people where talking about when they were making sad noises about the Hugos?
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p>(<em>Warning: some links lead to to triggery ranting</em>. As James D. Nicoll warns: "memetic prophylactic recommended".)</p> <p>By now, everybody who cares knows that the <a href="http://www.tor.com/blogs/2015/04/2015-hugo-award-nominees">nominations for the 2015 Hugo Awards</a> reflect the preferences of a <a href="https://bradrtorgersen.wordpress.com/2015/02/01/sad-puppies-3-the-2015-hugo-slate/">bloc-voting slate</a> with an <ins>agenda—and</ins> <del>agenda--and</del> <a href="http://voxday.blogspot.co.uk/2015/02/rabid-puppies-2015.html">their culture wars allies</a>. But, interestingly, a new Hugo-related record has been set: for a <a href="http://www.castaliahouse.com/">Finnish publisher few people have ever heard of</a> is responsible for no fewer than nine nominated works.</p> <p>Castalia House was (per wikipedia) founded by <a href="http://en.wikipedia.org/wiki/Theodore_Beale#Castalia_House">Theodore Beale (aka Vox Day)</a> in early 2014 in <a href="http://www.castaliahouse.com/ch-page/contact/">Kouvola, Finland</a>. As their website explains:</p> <blockquote><p>Castalia House is a Finland-based publisher that has a great appreciation for
the golden age of science fiction and fantasy literature. The books that we
publish honor the traditions and intellectual authenticity exemplified by
writers such as J.R.R. Tolkien, C.S. Lewis, Robert E. Howard, G.K.
Chesterton, and Hermann Hesse. We are consciously providing an alternative to
readers who increasingly feel alienated from the nihilistic, dogmatic science
fiction and fantasy being published today. We seek nothing less than a
Campbellian revolution in genre literature.</p> </blockquote> <p>Total culture wars, <a href="http://wehuntedthemammoth.com/2015/04/04/vox-day-christians-need-to-follow-the-lead-of-gamergate-and-the-spanish-inquisition-to-defeat-seculars-satan/">very gamergate</a>, much fail, wow. But the screaming question I feel the need to ask, is: why Finland? <a href="http://www.migranttales.net/of-birds-and-feathers-the-ps-the-sweden-democrats-and-their-american-bedfellows/">Could there be a connection</a> between the <a href="http://en.wikipedia.org/wiki/Finns_Party">white supremacist Perussuomalaiset</a> (Finns Party), the overtly racist <a href="http://en.wikipedia.org/wiki/Sweden_Democrats">Sweden Democrats</a>, the <a href="http://voxday.blogspot.nl/2014/01/dark-enlightenment-second-stage.html">Dark Enlightenment/neoreactionary</a> movement, and <a href="http://www.donotlink.com/framed?673157">Vox Day's peculiarly toxic sect</a> of Christian Dominionist theology?</p> <p>Vox Day writes:</p> <blockquote><p>It's time for the church leaders and the heads of Christian families to start
learning from #GamerGate, to start learning from Sad Puppies, and start
leading. Start banding together and stop accommodating the secular world in
any way. Don't hire those who hate you. Don't buy from those who wish to
destroy you. Don't work with those who denigrate your faith, your traditions,
your morals, and your God. Don't tolerate or respect what passes for their
morals and values.</p> </blockquote> <p>Over a period of years, he's built an international coalition, finding common cause with the European neo-nazi fringe. Now they've attempted to <a href="http://io9.com/the-hugo-awards-were-always-political-now-theyre-only-1695721604">turn the Hugo Awards into a battlefield</a> in their (American) culture wars. But this clearly isn't the end game they have in mind: it's only a beginning. (The Hugos, by their very nature, are an award anyone can vote in for a small fee: it is interesting to speculate on <a href="http://en.wikipedia.org/wiki/Robert_Beale_%28entrepreneur%29">how deep</a> Vox Day's pockets are.) But the real burning question is, "what will he attack next?"</p> <p>My guess: the Hugo awards are not remotely as diverse and interesting as the SFWAs <a href="https://www.sfwa.org/nebula-awards/">Nebula Awards</a> <ins>—an</ins> <del>--an</del> organization from which Vox Day became only the <a href="http://www.locusmag.com/News/2013/08/beale-expelled-from-sfwa/">second person ever</a> to be expelled. I believe he bears SFWA (and former SFWA President John Scalzi) no love, and the <a href="https://www.sfwa.org/about/join-us/sfwa-membership-requirements/">qualification for SFWA membership</a> (which confers Nebula voting rights) is to have professionally published three short stories or a novel. Castalia House is a publishing entity with a short story anthology series. Is the real game plan "Hugos today: Nebulas tomorrow?"</p>Fundamental Forces2015-02-26T21:23:02.995000Zhttp://xkcd.com/1489/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/fundamental-forces/5994357:d96dc5">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/5994357.png" style="vertical-align: middle;width:16px;height:16px;"> xkcd.com.</b>
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<img alt=""Of these four forces, there's one we don't really understand." "Is it the weak force or the strong--" "It's gravity."" src="http://imgs.xkcd.com/comics/fundamental_forces.png" title=""Of these four forces, there's one we don't really understand." "Is it the weak force or the strong--" "It's gravity."" />Control Panel2014-12-14T21:42:10.222000ZJason Benzhttp://jason.bennee.com/blog/2014/12/control-panel/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/control-panel/2081204:9628f5">shared this story</a>
from <img src="https://www.newsblur.com/rss_feeds/icon/2081204" style="vertical-align: middle;width:16px;height:16px;"> Benz World Blog:</b>
</td>
</tr>
<tr>
<td>
Nice aesthetic
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p>I replaced the control panel in <a href="http://jason.bennee.com/blog/2011/11/toyota-hiace/">our van</a>. The original <a href="http://jason.bennee.com/blog/2014/07/solar-controller/">solar controller</a> was used to recharge the <a href="https://www.youtube.com/user/musiclive4u">ML4U</a> film crew at <a href="http://jason.bennee.com/blog/tag/mitchell-creek/">Mitchell Creek</a> Rock n’ <a href="http://jason.bennee.com/blog/tag/blues/">Blues</a> <a href="http://jason.bennee.com/blog/tag/festival/">Festival</a> but needed some improvements. I added an ammeter, upgraded the volt meters and swapped power hungry <a href="http://jason.bennee.com/blog/tag/lights/">lights</a> for LED’s</p>
<div name="googleone_share_1" style="float: left; margin-left: 0px;"></div><br><br><img src="http://jason.bennee.com/blog/wp-content/uploads/2014/12/Mark-Two.jpg" />Offline reading and a dark theme on the Android app2014-11-20T08:24:06.576000Zhttp://blog.newsblur.com/post/102954904310<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/offline-reading-and-/6227690:862471">shared this story</a>
:</b>
</td>
</tr>
<tr>
<td>
Finally. It fixes the issues with Lollipop as well.
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p>This is a huge release for the NewsBlur Android app. So much has happened in the last six months. We’ve had a big backend rewrite that makes the app feel like new. It’s faster, which is necessary to make the new offline reading feature really shine. Now you can read on your Android phone or tablet while on the train, underground, or just in airplane mode to save battery.</p>
<p>Better yet, not only can you read while offline, but you can better read at night with the new dark theme. Take a look.</p>
<p><a href="https://www.newsblur.com/android"><img src="http://static.newsblur.com.s3.amazonaws.com/blog/android%20v4.png" style="width: 650px; margin: 0 auto;" /></a></p>
<p>Here’s what’s new in version 4.0:</p>
<ul><li>Offline reading and story syncing. Stories read while offline will sync as soon as you’re back online. </li>
<li>Option to also store images from stories for offline reading. They do take space, though.</li>
<li>A dark theme for reading at night.</li>
<li>The Text mode can be set to automatically fetch, so you can get to the full text quicker.</li>
<li>An option to mark stories newer than a story as read.</li>
<li>Major backend improvements for faster story loading and transitions. This app should feel like new.</li>
</ul><p>The NewsBlur Android app is a <a href="https://www.newsblur.com/android">free download</a>. And as always, there’s more big ticket features coming soon for Android.</p>German Scrabble2014-11-20T08:03:55.546000ZManuhttp://feedproxy.google.com/~r/bonkersworldnet/~3/zddYAdeouXg/<table style="border: 1px solid #E0E0E0; margin: 0; padding: 0; background-color: #F0F0F0" valign="top" align="left" cellpadding="0" width="100%">
<tr>
<td rowspan="2" style="padding: 6px;width: 36px;white-space:nowrap" width="36" valign="top"><img src="https://www.gravatar.com/avatar/a7d7f408c0b3370bbbeb98833d6c50e4" style="width: 36px; height: 36px; border-radius: 4px;"></td>
<td width="100%" style="padding-top: 6px;">
<b>
stsquad
<a href="https://stsquad.newsblur.com/story/german-scrabble/219084:e0c2e3">shared this story</a>
from <img src="https://s3.amazonaws.com/icons.newsblur.com/219084.png" style="vertical-align: middle;width:16px;height:16px;"> Bonkers World.</b>
</td>
</tr>
</table>
<hr style="clear: both; margin: 0 0 24px;">
<p><a href="http://www.bonkersworld.net/images/2014.11.11_german_scrabble.png"><img src="http://www.bonkersworld.net/images/2014.11.11_german_scrabble.png" /></a></p>
<p><!--
google_ad_client = "ca-pub-9761855105959184";
/* 728 x 90, 2011.07.13 */
google_ad_slot = "9028864387";
google_ad_width = 728;
google_ad_height = 90;
//-->
</p>
<img height="1" src="http://feeds.feedburner.com/~r/bonkersworldnet/~4/zddYAdeouXg" width="1" />