<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://josephcottam.github.io//somewhatuseful/feed.xml" rel="self" type="application/atom+xml" /><link href="https://josephcottam.github.io//somewhatuseful/" rel="alternate" type="text/html" /><updated>2026-01-13T21:33:41+00:00</updated><id>https://josephcottam.github.io//somewhatuseful/feed.xml</id><title type="html">Somewhat Useful</title><subtitle>Somewhatuseful is the most I hope for.</subtitle><entry><title type="html">Thoughts on Writings about AI</title><link href="https://josephcottam.github.io//somewhatuseful/2026/01/13/Thoughts-on-Writtings-about-AI.html" rel="alternate" type="text/html" title="Thoughts on Writings about AI" /><published>2026-01-13T00:00:00+00:00</published><updated>2026-01-13T00:00:00+00:00</updated><id>https://josephcottam.github.io//somewhatuseful/2026/01/13/Thoughts-on-Writtings-about-AI</id><content type="html" xml:base="https://josephcottam.github.io//somewhatuseful/2026/01/13/Thoughts-on-Writtings-about-AI.html"><![CDATA[<p>Recent advances in ML<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> have introduced a lot of new capabilities<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup>.  The degree and kind of change are driving a lot of interesting technology-and-society articles.  I’m collecting the ones that catch my attention here…</p>

<table>
  <thead>
    <tr>
      <th>Summary</th>
      <th>Commentary</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><a href="https://danielmiessler.com/blog/keep-the-robots-out-of-the-gym">Keep the Robots out of the Gym</a>: A simple rule for personally judging when AI should/not be used.  At “Work” the goal is to <em>move</em> a heavy object, and its OK to use a machine to do it.  In the “Gym” the goal is to <em>lift</em> a heavy object, and using a machine would defeat the purpose.</td>
      <td>This framing clarifies a lot of times that AI is/not appropriate: Is the goal for <em>me</em> to do or for it <em>to be done</em>?  Implicit in the discussion is that the AI does the thing well-enough to call it done…</td>
    </tr>
    <tr>
      <td><a href="https://www.experimental-history.com/p/why-arent-smart-people-happier">Why Aren’t Smart People Happier</a>: There is a section in the middle that notes that AI takes all problems and reduces them to well-defined problems…wether appropriate or not.  The annecdote that made me laugh: “If you booted up a super-smart AI in ancient Greece, fed it all human knowledge, and asked it how to land on the moon, it would respond ‘You can’t land on the moon. The moon is a god floating in the sky.’ How would you get it to realize the moon is actually a big rock? That’s a great, poorly defined problem, and I don’t expect AI to solve it anytime soon.”</td>
      <td>“Well defined problems” are those are repeatable, involve stable relationships and answers are unambiguous.  All other problems are “poorly defined”.<br /><br />The implication is that people will always be important for “poorly defined” problems.  I think also in the knowledge-blindspot space.<br /><br />I was also caught by the commentary that if we define all human/animal/plant as fitting under some category of “intelligence”, then we loose the power of the category. It is hard to treat in/out judgements as not high/low value, but that doesn’t mean the in/out judgement is not useful…just be careful with it!</td>
    </tr>
    <tr>
      <td><a href="https://simonwillison.net/2025/Oct/7/vibe-engineering/">Vibe Engineering</a>: The ability to <em>iterate</em> on code is what makes coding agents distinct.  Practice is converging toward running multiple agents at once on a codebase.  Agents work best when other software engineering practices are followed <sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">3</a></sup>: Automated testing, advance planning, documentation, version control, effective automation, code-review culture, “weird management” in the agile manifest sense (I think), non-automated QA practices, preview/integration testing. <br /><br /><strong>Observation</strong>: “AI tools amplify existing expertise”.</td>
      <td>Reminds me the concept that technology is a “force multiplier”.  <br /><br />I think the AI-thing is amplifying the impact of good and bad habits.  This will make some bad habits more painful, and possibly make them less common!</td>
    </tr>
    <tr>
      <td><a href="https://addyosmani.com/blog/next-two-years/">The next two years of Software Engineering</a>: Professional strategies need to shift, but its not clear what the end-game looks like yet.  This post goes through how a Junior and Senior dev might strategically act now for many possible futures.  <br /><br /><strong>Observation:</strong> Coding was never the sole task. Lean into those other tasks.<br /><br />Closely related is <a href="https://anniecherkaev.com/choosing-learning-over-autopilot">Choose Learning over Autopilot</a>, but this later piece is more tactically focused.</td>
      <td>The most thorough analysis that I’ve seen of what people making software could keep focus on that will be important in a variety of futures.<br /><br />There are concrete actions a software worker could take and those are spelled out here.  Though they mostly boil down to “Juniors, learn the fundamentals; Seniors, learn the new technology’ they are actually more concrete to address specific potential/plausible futures.</td>
    </tr>
  </tbody>
</table>

<p>Philosophy that has resonated:</p>
<ul>
  <li><a href="https://cacm.acm.org/blogcacm/why-nietzsche-matters-in-the-age-of-artificial-intelligence/">Why Nietzsche Matters in the Age of Artificial Intelligence</a></li>
  <li><a href="https://computingeducationthings.substack.com/p/22-dont-turn-your-brain-off">Don’t turn of your brain</a></li>
  <li><a href="https://davegriffith.substack.com/p/software-development-in-the-time">Software Development in the Time of Strange New Angels</a></li>
</ul>

<p>Things that look specifically at agentic workflows:t</p>
<ul>
  <li><a href="https://www.anthropic.com/engineering/advanced-tool-use">Tools on Claude</a></li>
  <li><a href="https://www.sawyerhood.com/blog/llm-extension">Bitter lesson LLM extensions</a></li>
</ul>

<div class="footnotes" role="doc-endnotes">
  <ol>
    <li id="fn:1" role="doc-endnote">
      <p>It’s often billed as advances in AI, but the subset of AI known as ML is the part that has improved. <a href="https://en.wikipedia.org/wiki/Artificial_intelligence">The wikipedia article on AI</a> paints AI with a very broad brush…taken at face value its hard to think of a computational field that is not AI.  Compilers involve resource planning (which is AI), programming languages are at least a knowledge representation problem (which is AI) and have heavy reasoning components in their optimization (which is AI). Cryptography or Computational theory seem safe for now…That being said, I don’t think <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:2" role="doc-endnote">
      <p>In 2019, new capabilities came from ‘deep models’, where you just do the neural-network thing but a thousand times in stead of three. In 2025 the advanced capabilities come from LLM, but I expect those to be passed sometime (though for marketing reasons the label may be with us for a lot longer than the language-based architectures). <a href="#fnref:2" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:3" role="doc-endnote">
      <p>Echoed in <a href="https://bits.logic.inc/p/ai-is-forcing-us-to-write-good-code">AI Is Forcing us to Write Good Code</a>, but the list of “best practices” is: 100% code coverage, Namespaces, Fast/Ephemeral/Concurrent dev environments, End-to-End Types.  Each list was drawn from the respective author’s personal experience an not from a formal education, appeal-to-authority or survey of peers.  In practice, a best practice is probably at least partially context dependent, and similarly the ones that AI will emphasize or deprecate will also change project to project.  That being said, both lists lean heavily on <em>infrastructure</em> practices. <a href="#fnref:3" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
  </ol>
</div>]]></content><author><name></name></author><category term="MachineLearning" /><category term="AI" /><summary type="html"><![CDATA[Recent advances in ML1 have introduced a lot of new capabilities2. The degree and kind of change are driving a lot of interesting technology-and-society articles. I’m collecting the ones that catch my attention here… It’s often billed as advances in AI, but the subset of AI known as ML is the part that has improved. The wikipedia article on AI paints AI with a very broad brush…taken at face value its hard to think of a computational field that is not AI. Compilers involve resource planning (which is AI), programming languages are at least a knowledge representation problem (which is AI) and have heavy reasoning components in their optimization (which is AI). Cryptography or Computational theory seem safe for now…That being said, I don’t think &#8617; In 2019, new capabilities came from ‘deep models’, where you just do the neural-network thing but a thousand times in stead of three. In 2025 the advanced capabilities come from LLM, but I expect those to be passed sometime (though for marketing reasons the label may be with us for a lot longer than the language-based architectures). &#8617;]]></summary></entry><entry><title type="html">Illustrating Decisions (in healthcare)</title><link href="https://josephcottam.github.io//somewhatuseful/2025/12/03/Illustrated-Decisions-healthcare.html" rel="alternate" type="text/html" title="Illustrating Decisions (in healthcare)" /><published>2025-12-03T00:00:00+00:00</published><updated>2025-12-03T00:00:00+00:00</updated><id>https://josephcottam.github.io//somewhatuseful/2025/12/03/Illustrated-Decisions-healthcare</id><content type="html" xml:base="https://josephcottam.github.io//somewhatuseful/2025/12/03/Illustrated-Decisions-healthcare.html"><![CDATA[<p><img src="/somewhatuseful/assets/images/healthcare_choices.png" alt="" /></p>

<p>I deeply enjoyed reading <a href="https://healthisotherpeople.substack.com/p/an-abominable-creature">this article</a> by <a href="https://substack.com/@andrewtsang">Andrew Tsang</a> that illustrates the flow of funds through different country’s healthcare systems.</p>

<p>What really stands out is how he phrases the analysis as uncovering the collective <em>choices and values</em> that the systems embody. This choice-based analysis works well with the visual metaphor in the <a href="https://en.wikipedia.org/wiki/Sankey_diagram">Sankey diagram</a>: each involves branching!</p>

<p>I like that the use of static images for the subsection call-outs in the <a href="https://healthisotherpeople.substack.com/i/175943797/observations-of-the-us-system-from-this-chart">observations on the US system</a>.  This is often done with with <a href="https://shorthand.com/the-craft/introduction-to-scrollytelling/index.html">scrollytelling</a>, using static images instead fits with my desires to flip between things in my own order.</p>

<p>To improve the diagram, I would try to stabilize the colors more: in the call-outs, sections are colored differently than they are in the combine diagram.  There is some standardization between sections (nursing homes are orange..usually) but I would use that more.  The worst offender is the diagram in the <a href="https://healthisotherpeople.substack.com/i/175943797/decision-workers-pay-at-least-twice">Workers pay at least twice</a>, which changes colors from the main diagram but never refers to any colors to guide you!  Then in the next section, the same colors are used for different things.  If the base diagram could be restructured to implicitly keep those colors, it would make it easier to relate each targeted diagram back to it.  I consider this a quibble, the diagrams remain understandable throughout.</p>

<hr />
<p>Header image originally sourced from <a href="https://healthisotherpeople.substack.com/p/an-abominable-creature">the source article</a>.  I could not find terms, so I’m the original author is OK with me posting a copy here and saying (again) that I think its an effective image and a good read.</p>]]></content><author><name></name></author><summary type="html"><![CDATA[]]></summary></entry><entry><title type="html">Neo4j: Export and Reload (updated)</title><link href="https://josephcottam.github.io//somewhatuseful/2025/12/03/Neo4J-Export-And-Load-updated.html" rel="alternate" type="text/html" title="Neo4j: Export and Reload (updated)" /><published>2025-12-03T00:00:00+00:00</published><updated>2025-12-03T00:00:00+00:00</updated><id>https://josephcottam.github.io//somewhatuseful/2025/12/03/Neo4J-Export-And-Load-updated</id><content type="html" xml:base="https://josephcottam.github.io//somewhatuseful/2025/12/03/Neo4J-Export-And-Load-updated.html"><![CDATA[<p>I had transfer a <a href="https://neo4j.com/">Neo4j</a> database to another team. They already had a database so I wanted to provide flexibility in how they imported and merged it.  Below are notes on options explored. Ultimately, none were obviously better than the others for all cases.</p>

<table>
  <thead>
    <tr>
      <th>Method</th>
      <th>td/dr</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Binary</td>
      <td>Good for a fresh database that you are an admin on.</td>
    </tr>
    <tr>
      <td>CSV <br />(one file)</td>
      <td>Export easy, import hard.<br />Requires pre-processing to do import.<br />If import/pre-processing is done wrong, silently drops data.</td>
    </tr>
    <tr>
      <td>CSV (bulk)</td>
      <td>Export limitations, import easy. <br />Viable when nodes have few label combinations.</td>
    </tr>
    <tr>
      <td>JSON</td>
      <td>Export easy, import limitation.<br />Slower than CSV. <br />Best used when there are few node labels.</td>
    </tr>
    <tr>
      <td>GML</td>
      <td>Silently drops data.</td>
    </tr>
    <tr>
      <td>XML</td>
      <td>Did not test…</td>
    </tr>
  </tbody>
</table>

<p>For context, here are the database stats:</p>
<ul>
  <li>2M nodes</li>
  <li>6M relationship</li>
  <li>&gt;3K distinct node labels</li>
  <li>&gt;10k distinct node label combinations</li>
  <li>&gt;600 distinct edge types</li>
</ul>

<p>My go to (in December 2025) is the JSON option:  It is fast-enough and has the fewest pitfalls, but be aware that JSON import can take an order of magnitude longer than export.</p>
<h2 id="binary-export">Binary export</h2>

<p><strong>Export:</strong> <code class="language-plaintext highlighter-rouge">neo4j-admin database dump neo4j --to-path /var/lib/neo4j</code>
<strong>Import:</strong> <code class="language-plaintext highlighter-rouge">neo4j-admin database load --from-path=datbase.dump neo4j</code> (Optionally, add <code class="language-plaintext highlighter-rouge">--overwrite-destination</code> if there is data.)</p>

<p><strong>Explanation:</strong> As near as I can tell, this is basically a near-exact copy of the database as it exists on disk.  <code class="language-plaintext highlighter-rouge">neo4j-admin</code> is a command line tool for administering the database.</p>

<p><strong>Pros:</strong></p>
<ul>
  <li>Binary export and import are fast (wall-clock time) and space efficient.</li>
  <li>The commands for neo4j-admin are <a href="https://neo4j.com/docs/operations-manual/current/backup-restore/offline-backup/">well documented</a></li>
  <li>It is a single command to both import and export</li>
  <li>Included in neo4j desktop tool in recent versions.</li>
</ul>

<p><strong>Cons:</strong></p>
<ul>
  <li>Requires stopping the database for both export and import.  This requires platform-specific commands.</li>
  <li>Cannot merge into existing database</li>
  <li>Binary dumps are tied to specific versions, so if your infrastructure doesn’t match it may not be viable.</li>
</ul>

<p><strong>Conclusion:</strong> If you have a fresh database, this a great option.</p>
<h2 id="csv-export-one-file">CSV export (one-file)</h2>
<p><strong>Export:</strong> <code class="language-plaintext highlighter-rouge">CALL apoc.export.csv.all("file:/database.csv", {});</code>
<strong>Import:</strong> Process into separate nodes &amp; relations files.</p>
<div class="language-cypher highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">CALL</span> <span class="n">apoc.import.csv</span><span class="ss">(</span>  
<span class="err">   </span> <span class="ss">[{</span><span class="py">fileName:</span> <span class="s1">'file:/database_nodes.csv'</span><span class="ss">,</span> <span class="py">labels:</span><span class="ss">[]}],</span>  
<span class="err">   </span> <span class="ss">[{</span><span class="py">fileName:</span> <span class="s1">'file:/import/database_relations.csv'</span><span class="ss">,</span> <span class="py">types:</span> <span class="err">‘’</span><span class="ss">}],</span>  
<span class="err">   </span> <span class="ss">{</span><span class="py">ignoreBlankString:</span> <span class="k">True</span><span class="ss">,</span> <span class="py">ignoreEmptyCellArray:</span> <span class="k">True</span><span class="ss">});</span>
<span class="k">MATCH</span><span class="w"> </span><span class="ss">()</span><span class="o">-</span><span class="ss">[</span><span class="n">t</span><span class="ss">]</span><span class="o">-</span><span class="ss">()</span> <span class="k">REMOVE</span> <span class="n">t.__csv_type</span><span class="ss">;</span>
<span class="k">MATCH</span><span class="w"> </span><span class="ss">(</span><span class="n">n</span><span class="ss">)</span> <span class="k">REMOVE</span> <span class="n">n.__csv_id</span><span class="ss">;</span>
</code></pre></div></div>
<p><strong>Explanation:</strong> The export produces a single csv with all of the columns needed to do both nodes and edges.  I have a script that loads up that file, splits it into nodes and edges based on the <code class="language-plaintext highlighter-rouge">_id</code> field, reformats the list of node labels to be semi-colon separated and renames columns to use the right names.  Including <code class="language-plaintext highlighter-rouge">ignoreBlankString</code> and , <code class="language-plaintext highlighter-rouge">ignoreEmptyCellArray</code> keeps the import from making all properties on all nodes/relations.</p>

<p><strong>Pros:</strong></p>
<ul>
  <li>Relatively quick</li>
  <li>Works when nodes have thousands of labels.</li>
  <li>The raw csv export and the post-processed csv are understood by other tools</li>
  <li>Some additional options in <a href="https://neo4j.com/docs/apoc/current/overview/apoc.export/apoc.export.csv.all/">apoc.export.csv.all</a> <a href="https://neo4j.com/docs/apoc/current/export/csv/">extended documentation</a> might make the process simpler.</li>
</ul>

<p><strong>Cons:</strong></p>
<ul>
  <li>The processing before input can be extensive.  I have not included my script here because it is tailored to my specific dataset.</li>
  <li>There are issues around list-typed and empty-valued fields.  It can take some non-trivial work to get an identical import after export+pre-processing.</li>
  <li>If you don’t do it right can <em>silently</em> drop of data on import</li>
</ul>

<p><strong>Conclusion:</strong> The post-processing has been a little tricky.  This is a flexible and faster-than-CSV option if you are comfortable making it.</p>

<h2 id="csv-export-bulk">CSV export (bulk)</h2>
<p><strong>Export</strong>: <strong>Export:</strong> <code class="language-plaintext highlighter-rouge">CALL apoc.export.csv.all("file:/database.csv", {bulkImport: true});</code>
<strong>Import:</strong></p>
<div class="language-cypher highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">CALL</span> <span class="n">apoc.import.csv</span><span class="ss">(</span>  
<span class="err">   </span> <span class="ss">[{</span><span class="py">fileName:</span> <span class="s1">'file:/database_&lt;label1&gt;.csv'</span><span class="ss">,</span> <span class="py">labels:</span><span class="ss">[]},</span>
<span class="err">   </span>  <span class="ss">{</span><span class="py">fileName:</span> <span class="s1">'file:/database_&lt;label2&gt;.csv'</span><span class="ss">,</span> <span class="py">labels:</span><span class="ss">[]}],</span>  
<span class="err">   </span> <span class="ss">[{</span><span class="py">fileName:</span> <span class="s1">'file:/import/database_&lt;type1&gt;.csv'</span><span class="ss">,</span> <span class="py">types:</span> <span class="err">‘’</span><span class="ss">},</span>
<span class="err">   </span> <span class="ss">{</span><span class="py">fileName:</span> <span class="s1">'file:/import/database_&lt;type2&gt;.csv'</span><span class="ss">,</span> <span class="py">types:</span> <span class="err">‘’</span><span class="ss">}],</span>
<span class="err">   </span> <span class="ss">{</span><span class="py">ignoreBlankString:</span> <span class="k">True</span><span class="ss">,</span> <span class="py">ignoreEmptyCellArray:</span> <span class="k">True</span><span class="ss">})</span>
</code></pre></div></div>
<p><strong>Explanation:</strong> <a href="https://neo4j.com/docs/apoc/current/overview/apoc.export/apoc.export.csv.all/">apoc.export.csv.all</a> with <code class="language-plaintext highlighter-rouge">bulkImport: true</code> creates a separate file for each node label combination and each relation type found in the database. You need to know those names at import time.</p>

<p><strong>Pros:</strong></p>
<ul>
  <li>Fast export</li>
  <li>Data is immediately ready for import</li>
</ul>

<p><strong>Cons:</strong></p>
<ul>
  <li>No generic import command: you need to know the files produced.</li>
  <li>Having nodes with many labels can fail because the files are split by label <em>combination</em> and the name length may exceed filesystem limits.</li>
</ul>

<p><strong>Conclusion:</strong> When it works, this is the simplest option.  Applicability depends on data characteristics.</p>

<p><strong>Note:</strong> The same trick used in JSON for many-node-labels might work here to. I ran out of time to investigate!</p>
<h2 id="json-export">JSON export</h2>
<p><strong>Export</strong>:</p>
<div class="language-cypher highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">MATCH</span><span class="w"> </span><span class="ss">(</span><span class="n">n</span><span class="ss">)</span> <span class="k">SET</span> <span class="n">n.labels</span> <span class="o">=</span> <span class="nf">labels</span><span class="ss">(</span><span class="n">n</span><span class="ss">);</span>
<span class="k">MATCH</span><span class="w"> </span><span class="ss">(</span><span class="n">n</span><span class="ss">)</span> <span class="k">CALL</span> <span class="n">apoc.create.removeLabels</span><span class="ss">([</span><span class="n">n</span><span class="ss">],</span> <span class="nf">labels</span><span class="ss">(</span><span class="n">n</span><span class="ss">))</span> <span class="k">YIELD</span> <span class="n">node</span> <span class="n">FINISH</span><span class="ss">;</span>
<span class="k">MATCH</span><span class="w"> </span><span class="ss">(</span><span class="n">n</span><span class="ss">)</span> <span class="k">SET</span> <span class="nl">n</span><span class="dl">:</span><span class="m">_</span><span class="n">IS_NODE</span><span class="ss">;</span>
<span class="k">CALL</span> <span class="n">apoc.export.json.all</span><span class="ss">(</span><span class="s2">"file:/database.json"</span><span class="ss">,</span> <span class="ss">{</span><span class="py">compression:</span> <span class="s2">"GZIP"</span><span class="ss">});</span>
</code></pre></div></div>
<p><strong>Import:</strong></p>
<div class="language-cypher highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">CREATE</span> <span class="k">CONSTRAINT</span> <span class="n">is_node_unique</span> <span class="n">IF</span> <span class="ow">NOT</span> <span class="ow">EXISTS</span> <span class="n">FOR</span><span class="w"> </span><span class="ss">(</span><span class="nl">n</span><span class="dl">:</span><span class="m">_</span><span class="n">IS_NODE</span><span class="ss">)</span> <span class="n">REQUIRE</span> <span class="n">n.neo4jImportId</span> <span class="k">IS</span> <span class="k">UNIQUE</span><span class="ss">;</span>
<span class="k">CALL</span> <span class="n">apoc.import.json</span><span class="ss">(</span><span class="s2">"file:/database.json"</span><span class="ss">);</span>
<span class="k">match</span><span class="w"> </span><span class="ss">(</span><span class="n">n</span><span class="ss">)</span> <span class="k">CALL</span> <span class="n">apoc.create.addLabels</span><span class="ss">(</span><span class="n">n</span><span class="ss">,</span> <span class="n">n.labels</span><span class="ss">)</span> <span class="k">YIELD</span> <span class="n">node</span> <span class="n">FINISH</span><span class="ss">;</span>
<span class="k">match</span><span class="w"> </span><span class="ss">(</span><span class="n">n</span><span class="ss">)</span> <span class="k">REMOVE</span> <span class="n">n.labels</span><span class="ss">;</span>
<span class="k">match</span><span class="w"> </span><span class="ss">(</span><span class="n">n</span><span class="ss">)</span> <span class="k">REMOVE</span> <span class="nl">n</span><span class="dl">:</span><span class="m">_</span><span class="n">IS_NODE</span><span class="ss">;</span>
<span class="k">MATCH</span><span class="w"> </span><span class="ss">(</span><span class="n">n</span><span class="ss">)</span> <span class="k">REMOVE</span> <span class="n">n.neo4jImportId</span><span class="ss">;</span>
<span class="k">DROP</span> <span class="k">CONSTRAINT</span> <span class="n">is_node_unique</span> <span class="n">IF</span> <span class="ow">EXISTS</span><span class="ss">;</span>
</code></pre></div></div>

<p><strong>Explanation:</strong> JSON import requires a uniqueness constraint for every node label.  When working with many of labels, this is impractical.  For export, this method moves all labels to a property and creates on label used by all nodes.  The process is reversed on import.  The use of <code class="language-plaintext highlighter-rouge">FINISH</code> when manipulating lables reduces memory requirements.</p>

<p><strong>Pros:</strong></p>
<ul>
  <li>Produces JSON which can be imported by a number of tools</li>
  <li><a href="https://neo4j.com/docs/apoc/current/overview/apoc.export/apoc.export.json.all/">apoc.export.json.all</a> can compress on export or input.  I compressed on export using but decompressed on the command line before import (as shown above).</li>
</ul>

<p><strong>Cons:</strong></p>
<ul>
  <li>Slowest import of all methods checked (5x slower than the CSV).  This is likely because it builds an index as it imports.</li>
  <li>The multi-step dance is easy to make a mistake during.  If you delete the labels property before successfully creating the node labels, it can cost a lot of time.</li>
</ul>

<p><strong>Conclusion:</strong> Its an OK process.  If there are only a few node labels, creating uniqueness constrains for each is preferable to the technique used here.</p>

<h2 id="other-notes">Other notes:</h2>
<ul>
  <li>Neo4j <code class="language-plaintext highlighter-rouge">apoc.import.csv</code> supports “<a href="https://neo4j.com/docs/operations-manual/current/import/#import-tool-id-spaces">id spaces</a>”.  The original database was built from multiple different sources providing different node types.  Id spaces made it easier to merge those sources together since we did not need to ensure the ids were disjoint between sources.</li>
  <li>The GML export dropped significant portions of the data without warning.  I would not advise using this option.</li>
  <li>We did not test the XML option because we ran out of time!</li>
</ul>]]></content><author><name></name></author><category term="graphs" /><category term="Databases" /><summary type="html"><![CDATA[I had transfer a Neo4j database to another team. They already had a database so I wanted to provide flexibility in how they imported and merged it. Below are notes on options explored. Ultimately, none were obviously better than the others for all cases.]]></summary></entry><entry><title type="html">Neo4j: Export and Reload</title><link href="https://josephcottam.github.io//somewhatuseful/2025/10/01/Neo4J-Export-And-Load.html" rel="alternate" type="text/html" title="Neo4j: Export and Reload" /><published>2025-10-01T00:00:00+00:00</published><updated>2025-10-01T00:00:00+00:00</updated><id>https://josephcottam.github.io//somewhatuseful/2025/10/01/Neo4J-Export-And-Load</id><content type="html" xml:base="https://josephcottam.github.io//somewhatuseful/2025/10/01/Neo4J-Export-And-Load.html"><![CDATA[<p>(Please see <a href="https://josephcottam.github.io/somewhatuseful/2025/12/03/Neo4J-Export-And-Load-updated.html">Neo4J-Export-And-Load-updated</a> instead.)</p>]]></content><author><name></name></author><category term="graphs" /><category term="Databases" /><summary type="html"><![CDATA[(Please see Neo4J-Export-And-Load-updated instead.)]]></summary></entry><entry><title type="html">Data types vs. Resource Types</title><link href="https://josephcottam.github.io//somewhatuseful/2025/08/21/Resources-and-Data.html" rel="alternate" type="text/html" title="Data types vs. Resource Types" /><published>2025-08-21T00:00:00+00:00</published><updated>2025-08-21T00:00:00+00:00</updated><id>https://josephcottam.github.io//somewhatuseful/2025/08/21/Resources-and-Data</id><content type="html" xml:base="https://josephcottam.github.io//somewhatuseful/2025/08/21/Resources-and-Data.html"><![CDATA[<p><a href="https://without.boats/">Without Boats</a> revisited ideas about a <a href="https://without.boats/blog/revisiting-a-smaller-rust/">“smaller rust”</a>.  The section on Resource types vs. Data types was very instructive.  The core distinction is that data can be copied while resources are shared.  This distinction is <em>lost</em> programming languages (Rust included) because the language uses a metaphor of RAM (which is a resource).</p>

<p>However, when you try to interpret different parts of memory it breaks into two major categories:</p>
<ul>
  <li><strong>Data</strong>: Immutable, meaning is derived entirely from context, separate copies are indistinguishable and interchangeable.</li>
  <li><strong>Resources</strong>: Maybe mutable, carry some inherent meaning and copies might be distinguishable from each other (minimally because mutability may lead them to have different values over time).</li>
</ul>

<p>Attending to this distinction enables some optimizations and freedoms that are often lost.</p>

<p>The post is instructive in how it illustrates that the distinction is latent in the Rust type system.  <code class="language-plaintext highlighter-rouge">Copy</code> items are data.  Any time you use a mutable borrow (<code class="language-plaintext highlighter-rouge">&amp;mut</code>), you are treating something like a resource.  Rust isn’t <em>enforcing</em> a data/resource distinction, it assumes everything is a Resource, but it <em>manifests</em> one and that might inform future language design.  This is in contrast to clojure that assumes everything is data and goes to some lengths to ensure that most resources behave like that through extensive use of persistent data structures, to get a resource you use the <code class="language-plaintext highlighter-rouge">transient</code> or <code class="language-plaintext highlighter-rouge">!</code> function variants, but it is presented as n optimization.</p>]]></content><author><name></name></author><category term="programming" /><summary type="html"><![CDATA[Without Boats revisited ideas about a “smaller rust”. The section on Resource types vs. Data types was very instructive. The core distinction is that data can be copied while resources are shared. This distinction is lost programming languages (Rust included) because the language uses a metaphor of RAM (which is a resource).]]></summary></entry><entry><title type="html">Reflections on the Bitter (and the Better) Lesson</title><link href="https://josephcottam.github.io//somewhatuseful/2024/10/12/The-Bitter-and-The-Better-Lesson.html" rel="alternate" type="text/html" title="Reflections on the Bitter (and the Better) Lesson" /><published>2024-10-12T00:00:00+00:00</published><updated>2024-10-12T00:00:00+00:00</updated><id>https://josephcottam.github.io//somewhatuseful/2024/10/12/The-Bitter-and-The-Better-Lesson</id><content type="html" xml:base="https://josephcottam.github.io//somewhatuseful/2024/10/12/The-Bitter-and-The-Better-Lesson.html"><![CDATA[<blockquote>
  <p>“…general methods that leverage computation are ultimately the most effective, and by a large margin.”
–Rich Sutton (March 13, 2019)</p>
</blockquote>

<p><a href="http://www.incompleteideas.net/IncIdeas/BitterLesson.html">The Bitter Lesson</a> is a short essay by Rich Sutton.  It’s worth a read, but  I’ll summarize the last two paragraphs here:</p>
<ul>
  <li>Observation S1: Attempts to inject “knowledge” leads to early advances but rapid plateaus while search and learning compound.</li>
  <li>Observation S2: There are two things that scale easily when you add more computation.  These same two things are driving current advances in AI.  Those two things are Search and Learning.</li>
  <li>Implication S1: General-purpose methods (like search &amp; learning) that scale with computation are the most valuable</li>
  <li>Implication S2: Minds are complex enough that seeking simplifications is ultimately a dead-end.  (This last one is a bit of a logical jump… but a smaller jump in Sutton’s essay than I’ve just wrriten here.)</li>
</ul>

<p>Sutton was talking specifically about AI advances, and how the past 6-ish years of deep learning punctuate his point.  Rodney Brooks posted a rebuttal <a href="https://rodneybrooks.com/a-better-lesson/">The Better Lesson</a>. I summarize Brook’s points as this:</p>
<ul>
  <li>Observation B1: Convolutional Neural Networks, neural network architectures and training regimes that preserve human-relevant factors are ways that humans still inject “knowledge”</li>
  <li>Observation B2: People need less than computers to learn these sorts of things so “something” is still missing.</li>
  <li>Observation B3: The <a href="http://www.gotw.ca/publications/concurrency-ddj.htm">free lunch is over</a> so don’t rely on it to improve your naive algorithms.</li>
</ul>

<p>Now for my (admittedly 5 years late) observations:</p>
<ul>
  <li>Brooks makes a strong point that “something” is missing, but this point is weakened because the place and method of knowledge-injection is radically different from Sutton’s.  Brook’s phrasing is a <strong>denial</strong> of Sutton’s thesis, when I see Brook’s arguments as fundamentally supporting Sutton’s point (that injecting knowledge at algorithm level is ultimately a loosing strategy).  Brooks and Sutton BOTH are arguing that we should look for smarter, more scalable ways to introduce things that we have learned are important. (But they disagree on what “scaleable” means.)</li>
  <li>Insofar as AI is trying to model a mind (human or otherwise), Implication S2 is probably in the right direction (though perhaps a bit fatalist).  However, when AI is trying to model/estimate something it must be assessed on a case-by-case basis. For example, physics-informed machine-learning probably still benefits from injecting simplified models (albiet, used in a different way with deep learning systems than with others).</li>
  <li>Search &amp; learning are general-purpose computational methods that scale for AI.  Does the observation hold for other parts of computer science?  I feel echos of RISC architectures, which under-pine the modern desktop systems…but do not feel as simple as they used to.  The scaling doesn’t always play out like we think, though the quest for simplicity will often be beneficial.</li>
  <li>Observation B1 really only works because the architectures created can be searched efficiently (relative to other stuff we’ve tried) for weights that effective. Colloquially, that type of search is called “learning”.  The search/learning are facilitated by the architecture….maybe B1 and S1 are the same observation in different clothing.</li>
</ul>]]></content><author><name></name></author><summary type="html"><![CDATA[“…general methods that leverage computation are ultimately the most effective, and by a large margin.” –Rich Sutton (March 13, 2019)]]></summary></entry><entry><title type="html">Automation and errors</title><link href="https://josephcottam.github.io//somewhatuseful/2024/10/10/Automation.html" rel="alternate" type="text/html" title="Automation and errors" /><published>2024-10-10T00:00:00+00:00</published><updated>2024-10-10T00:00:00+00:00</updated><id>https://josephcottam.github.io//somewhatuseful/2024/10/10/Automation</id><content type="html" xml:base="https://josephcottam.github.io//somewhatuseful/2024/10/10/Automation.html"><![CDATA[<blockquote>
  <p>Most all automations are not defined by the standard case or the typical inputs but by exceptions.
– Steven Sinofsky</p>
</blockquote>

<p>In some ways, programming is all about automation.  BUT there are some tasks that are more clearly about it than others.  While reading about the philosophy of various build/data-pipeline systems, I came across <a href="https://hardcoresoftware.learningbyshipping.com/p/222-automating-processes-with-software">this post</a> about what drives complexity in such systems.  It’s not the “happy path”, its the “sad path”…because there are usually so many sad paths and so many ways to get onto them!</p>

<p>A second point the article drives home is that the exception/error cases are usually discovered incrementally through use.  They almost by definition can’t be built in from the start because they often <em>don’t even exist</em> when the system is first created.</p>

<p>Having recently helped my daughter get her driver’s license, the point about many human-centered processes being focused on exceptions really drove home.  I have renewed my licence many times through the years.  I use a mail in form (anciently) or electronic form (recently).  But a FIRST license is an <em>exception</em> case…and required a trip the BMV and standing in line…and going back three times for various bureaucratic (i.e., exceptional case) reasons.</p>

<p>The observation at the end of the article is that many successful computer tools help people to <em>finish</em> a job rather than <em>start</em> a job. The folks I know who are powerpoint/illustrator/chatGPT are capable, knew their inputs, are well versed in the non-computer skills relevant to presenting/graph-design/writing.   They <em>could</em> do it themselves, but computation/automation helps them do it faster.  The people that <em>don’t</em> have those baseline skills….no amount of powerpoint wizardry, illustrator fiddling or ChatGPT prompting will save their presentation/drawing/novel.</p>]]></content><author><name></name></author><summary type="html"><![CDATA[Most all automations are not defined by the standard case or the typical inputs but by exceptions. – Steven Sinofsky]]></summary></entry><entry><title type="html">Git Repos for Pip install</title><link href="https://josephcottam.github.io//somewhatuseful/2024/10/02/Git-and-Pip.html" rel="alternate" type="text/html" title="Git Repos for Pip install" /><published>2024-10-02T00:00:00+00:00</published><updated>2024-10-02T00:00:00+00:00</updated><id>https://josephcottam.github.io//somewhatuseful/2024/10/02/Git-and-Pip</id><content type="html" xml:base="https://josephcottam.github.io//somewhatuseful/2024/10/02/Git-and-Pip.html"><![CDATA[<p>I use a small collection of python utilities I developed across several projects. Today, I started an experiment sharing those utilities between projects using pip to install a utilities package from a git repo.</p>

<p>I used to just copy my utilities and modify them in place to my hearts content BUT they have largely stabilized (so they change slowly) and most changes are now bug-fixes (so they need to be propagated to other projects).</p>

<p>For example, I have a tool that lets me edit JSON files using python lists and dictionaries in a path-like syntax.  I find it more convenient than using <a href="https://en.wikipedia.org/wiki/JSONPath">jsonpath </a>for queries AND it lets me edit things in a way similar to <a href="https://glom.readthedocs.io/en/stable/">glom</a> but covering a very important case that <a href="https://stackoverflow.com/questions/66377718/glom-assign-using-a-check-filter">I could not get glom to do</a>.  In the process of making my edit-tool, I DID NOT cover a lot of cases that glom does…so this is very much a tool-for-me…but it is a tool I’m using in 5 different projects now.</p>

<p>Pretending my tool is called <code class="language-plaintext highlighter-rouge">json_tool</code> and pretending that its git repo url is <code class="language-plaintext highlighter-rouge">ssh://git@fancyserver.org:somewhatuseful/json_tool.git</code>…</p>

<p>The repo structure follows the current practice of a <code class="language-plaintext highlighter-rouge">pyproject.toml</code> file and the package source files in a directory with the same name as the project (<code class="language-plaintext highlighter-rouge">json_tool</code> today).</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>root
├─ json_tool
|  ├─ __init__.py
|  ├─ find.py 
|  └─ edit.py
└─ pyproject.toml
</code></pre></div></div>

<p>The package source directory (and all other directories under it…if they existed) has an <code class="language-plaintext highlighter-rouge">__init__.py</code> to indicate they are <a href="https://docs.python.org/3/reference/import.html#regular-packages">regular packages</a> (I did not test with namespace packages).</p>

<p>To install, <code class="language-plaintext highlighter-rouge">pip install git+&lt;normal-repo-checkout-url&gt;</code>.  If the package name (specified as <code class="language-plaintext highlighter-rouge">name=&lt;package-name&gt;</code> in <code class="language-plaintext highlighter-rouge">pyproject.toml</code>) does not match repository name, you <em>might</em> use  at the end of the URL used in <code class="language-plaintext highlighter-rouge">pip install git+&lt;URL&gt;#egg=&lt;package-name&gt;</code> for the install. (This fragment fixed a problem…that I couldn’t reproduce using the fragment-free URL after it was fixed once.)</p>

<p>To update after bug-fixes, I use <code class="language-plaintext highlighter-rouge">pip install --force-reinstall --no-deps git+&lt;normal-repo-checkout-url&gt;</code>.  This forces the re-install of my custom package, but does not stomp on the other dependencies in my environment.  (The <code class="language-plaintext highlighter-rouge">--no-deps</code> might be removed in the future as I learn more about proper control of dependencies in <code class="language-plaintext highlighter-rouge">pyproject.toml</code> or if I-learn-how/conda-gains-the-ability to install from git in a way that isn’t just calling pip under the covers…)</p>]]></content><author><name></name></author><summary type="html"><![CDATA[I use a small collection of python utilities I developed across several projects. Today, I started an experiment sharing those utilities between projects using pip to install a utilities package from a git repo.]]></summary></entry><entry><title type="html">Software Artifact Visualization</title><link href="https://josephcottam.github.io//somewhatuseful/2024/07/16/Software-Artifact-Visualization.html" rel="alternate" type="text/html" title="Software Artifact Visualization" /><published>2024-07-16T00:00:00+00:00</published><updated>2024-07-16T00:00:00+00:00</updated><id>https://josephcottam.github.io//somewhatuseful/2024/07/16/Software-Artifact-Visualization</id><content type="html" xml:base="https://josephcottam.github.io//somewhatuseful/2024/07/16/Software-Artifact-Visualization.html"><![CDATA[<p>I recently read this blog post on “<a href="https://blog.sbensu.com/posts/demand-for-visual-programming/">visual programming</a>”.  The authors start by deriding syntax-based visualization (which is basically fair) and go on to argue for context and execution artifacts to support traditional programming.</p>

<p>The article is called “visual programming” but it only treats construction-of-a-program-via-visual-means superficially.  The only type of visual programming examined is basically alternate-syntax.  The issues that they point out are fair, but it not really a look at visual programming.</p>

<p>That being said, their argument to <em>support</em> traditional programming through visuals caught my attention.</p>
<h2 id="context">Context</h2>
<p>How is this codebase structured now and how has it evolved?  What has been retained?  What is it related to?  Examples used include:</p>
<ul>
  <li>SourceTrail: Sort of an automated UML-class diagram made automatically.  The strength the author points out is that it is presented NEXT TO the source code, so it is truly <strong>context</strong> for source editing.  Used in that way, and automatically generated, I think this is a good idea.</li>
  <li>Code retention diagram: How long has this been around?  Though the shelf-life of code itself is not one of my particular interests, looking at historical information in this way may help understand who to talk to or how careful to be with some code regions.</li>
  <li>Computer-network/service-topology: For slices of a network, I have found these valuable in my work.  For small networks they solve a navigation problem that is hard to trace out the json files they come from: What is interconnected? (I don’t think they scale to large networks well, I’ve never tried!)</li>
  <li>Treemaps of source code:  This is at the bottom of my list because I don’t find these diagrams terribly readable or compelling.  The observations being made are usually achievable by sorting a data table appropriately and reading the top 3(+/-2) lines.</li>
</ul>

<h2 id="execution-artifacts">Execution Artifacts</h2>

<p>How does the codebase achieve its intended result?  NOT the result itself, but the process that led up to that result. Examples used included:</p>
<ul>
  <li>protocol visualizations</li>
  <li>“swim-lane” diagrams</li>
  <li>State machines</li>
  <li>Memory layout (but not global, for select data structures)</li>
  <li>Memory traces (but not global, for selected values)</li>
  <li>Value ownership</li>
</ul>

<p>I think this is an important area.  The amount time/mental-effort it often takes to move program-execution trace information to a visual often exceeds the time/mental-effort to use a step debugger (except in parallel/distributed systems).  Reducing the effort to cross this bridge is a recurring daydream…but then I need to go do my actual job.</p>]]></content><author><name></name></author><summary type="html"><![CDATA[I recently read this blog post on “visual programming”. The authors start by deriding syntax-based visualization (which is basically fair) and go on to argue for context and execution artifacts to support traditional programming.]]></summary></entry><entry><title type="html">Tests are post-hoc brittleness</title><link href="https://josephcottam.github.io//somewhatuseful/2024/04/16/Brittle-Tests-and-Types.html" rel="alternate" type="text/html" title="Tests are post-hoc brittleness" /><published>2024-04-16T00:00:00+00:00</published><updated>2024-04-16T00:00:00+00:00</updated><id>https://josephcottam.github.io//somewhatuseful/2024/04/16/Brittle-Tests-and-Types</id><content type="html" xml:base="https://josephcottam.github.io//somewhatuseful/2024/04/16/Brittle-Tests-and-Types.html"><![CDATA[<p>In their <a href="https://journal.infinitenegativeutility.com/leaving-haskell-behind">post on leaving Haskell behind</a>, the term “brittle” is used as a compliment to mean things break in obvious ways. This is in contrast to the derogatory sense that “brittle” often takes in software, rough “unable to easily adapt to a new use case”.  I like the idea that brittle is a not a judgement but a <em>property</em> and how it is manifests (or is used) in a system determines if it is a good or bad thing.</p>

<p>Wikipedia says “A material is brittle if, when subjected to stress, it fractures with little elastic deformation and without significant plastic deformation.”  That is to say, brittle things won’t bend, they just break.</p>

<p>Fuses are an example where brittleness is used to benefit an electrical system.  Rather than slowly degrading, they trip…and the whole system stops before worse things can happen.</p>

<p>In Haskell, types introduce useful brittleness.  They are compiler checked assertions about the program.  If the types don’t match up, the system stops.  In Python, tests are a deliberately brittle point (when written regularly, run often and issues routinely addressed)<sup id="fnref:optionalTypes" role="doc-noteref"><a href="#fn:optionalTypes" class="footnote" rel="footnote">1</a></sup></p>

<p>The trick is to hit the <em>right</em> amount of brittleness in the right places.  What constitutes “stress” to the system?  Where can breaking under stress prevent worse issues?</p>

<hr />
<div class="footnotes" role="doc-endnotes">
  <ol>
    <li id="fn:optionalTypes" role="doc-endnote">
      <p>The introduction of types into python as <em>optionally</em> checked makes this parallel between types and test even more apparent.  The idea that types are a test-suite that the compiler checks has always resonated with me, though I don’t think its the whole story of types. <a href="#fnref:optionalTypes" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
  </ol>
</div>]]></content><author><name></name></author><summary type="html"><![CDATA[In their post on leaving Haskell behind, the term “brittle” is used as a compliment to mean things break in obvious ways. This is in contrast to the derogatory sense that “brittle” often takes in software, rough “unable to easily adapt to a new use case”. I like the idea that brittle is a not a judgement but a property and how it is manifests (or is used) in a system determines if it is a good or bad thing.]]></summary></entry></feed>