1. How a this better than just using any other script language?
2. Inventing a new language complicates large models ability to generate such scrips compared to a well-known language. Did you find it to be a problem? How did you mitigate?
3. The AI is showing. :) I had the similar discussion with ChapGPT and some phrasing is near the same. Not a dig, just a funny observation.
4. Consider the recursive nature of the problem you’re solving - large model updates workflow which you review each time, worker models generate plans and tool calls which you don’t review. A constrained language is useful in both cases to guide the model.
5. This Earlier discussion can provide useful background for why this is needed. You have probably seen it, but the readers will likely appreciate. https://news.ycombinator.com/item?id=48051562
The problem is real, Thank you for taking a stab and sharing your findings.
sshwarts 2 hours ago [-]
I really appreciate the comments and the encouragement.
"How <is> this better than just using any other script language?"
I wanted to make the language something built for a machine to write and a human to approve with a narrow scope and my control of what extends that. What shell commands, if any. What MCP and what tools in that MCP, and so on. If an agent generates a Python script to run unattended every morning, I feel I can't control it.
"Inventing a new language complicates large models ability to generate such scripts..."
I tried to keep it as small as possible and borrow a lot from what I knew models had seen in makefiles, YAML-ish, etc.
That was actually the question I began with. What would a make file look like if I were using it to make a classic markdown skill? Asking that question, sort of spawned the rest of it.
I tested the language against cold agents in both frontier model and local model camps to see where they had difficulty nad made changes as required. The lint approach as well as help topics from the MCP helped a lot.
"AI Showing..."
Yea there is a certain irony there. I freely admit product was written with Claude Code. I wrote the Product Requirements & Engineering Requirements and reviewed the code, so I can say I own it.
"Consider the recursive nature of the problem you’re solving..."
That framing is better than mine. When a frontier model writes a skillscript, there's not much chance of something slipping by. Over time, I've become cautiously open to the model adapting the skillscript to solve problems as they occur. Example, a skillscript runs every morning and checks github for PRs and issues. The agent when woken after the run 'noticed' an issue, fixed the script and it was ready for me to approve in the morning. Yes, that could have been done in Python, but I'd not be as confident about it.
I also really appreciate the link.
twalla 3 hours ago [-]
It seems like you’re dramatically overcomplicating what could be a 2 bash scripts and a single LLM call. Natural language and LLMs are great for searching the problem space to find a solution, once you find the solution, shrink the stochastic parts (the MD saying “check GitHub” or whatever) and grow the deterministic parts (a bash script) as much as possible.
sshwarts 1 hours ago [-]
That's a fair comment. I sorta feel you're describing my thesis, "shrink the stochastic, grow the deterministic" is the point. Where we differ, maybe, is what the deterministic part is made of.
The problem I'm trying to solve for me is a step removed. The agent is the one writing the script, and I want it to run unattended against my systems every morning. I can't let it write and run bash. Even though my agent lives in a container (NanoClaw), I still need it to reach out to other systems. Github, my other servers, MCPs, etc.
Skillscript is how I let it reach out without holding the keys. It can invoke a skillscript that hits GitHub, but it never runs the raw command or holds the token itself. The runtime holds the credential and only lets it through the skillscript I approved.
It's fair to say a skillscript basically is your two bash scripts and an LLM call, but fenced in.
dlahoda 5 hours ago [-]
In Brave, on stock Google Pixel 10, guard enabled, docs site layout is broken.
sshwarts 3 hours ago [-]
Thanks. I'm guessing since the docs are mintlify hosted, the CDN assets might be being blocked. I'll look and see if adding a domain sidesteps it.
bpavuk 4 hours ago [-]
same, Zen Browser (Firefox fork) on Linux
sshwarts 2 hours ago [-]
Is what you are seeing is the topics sidebar overlapping the text?
DonHopkins 5 hours ago [-]
[flagged]
Zetaphor 5 hours ago [-]
Brave is Chromium, presumably this means the site is broken on mobile chrome
sieabahlpark 4 hours ago [-]
[dead]
DonPlaton 2 hours ago [-]
[flagged]
DonHopkins 6 hours ago [-]
[flagged]
spankalee 5 hours ago [-]
LLMs are fantastic at generating new languages given docs and examples.
DonHopkins 2 hours ago [-]
Generating but not programming in them. They need to be reminded of the complete language definition they generated in every prompt, which is extremely costly, inefficient, and ultimately pointless, since any language you make up can't hold a candle to Python and its ecosystem, because it doesn't have an ecosystem, and the language itself doesn't exist in the training data.
How can you not get that? Do you believe LLMs remember what you show them between calls? That's not how they work. Each call starts from a clean slate, you have to re-describe the new language each and every call. There's no way to get around that. They are not magic. They do not learn from your prompts, which have absolutely no effect on the model itself.
If you think they do, you are falling for an illusion. ChatGPT is appending each of your incremental prompts to the full prompt, and it grows and grows longer and longer every time you add something. Sure, it summarizes when the full prompt gets to long, but that makes it distort and forget your language definition, and you have to add it again. If you give it the prompt to generate the language from scratch each time instead of the generated language itself, it generates a different language every time. You can't "cleverly hack" or "wish" your way out of that.
They may be good at generating new languages, but one thing that LLMs aren't good at apparently is warning you it's futile to generate a new language intended for llms to program instead of just using existing languages. They just play along and do ridiculous useless things out of syncophancy.
dlahoda 5 hours ago [-]
Not sure I understood well your comment.
Do you propose just ask AI to generate orchestration in Python?
DonHopkins 5 hours ago [-]
[flagged]
deadbabe 4 hours ago [-]
So no new languages ever?
DonHopkins 2 hours ago [-]
That's not what I said at all.
deadbabe 33 minutes ago [-]
Do you want to describe what the exception is to what you said then?
Rendered at 21:41:36 GMT+0000 (Coordinated Universal Time) with Vercel.
2. Inventing a new language complicates large models ability to generate such scrips compared to a well-known language. Did you find it to be a problem? How did you mitigate?
3. The AI is showing. :) I had the similar discussion with ChapGPT and some phrasing is near the same. Not a dig, just a funny observation.
4. Consider the recursive nature of the problem you’re solving - large model updates workflow which you review each time, worker models generate plans and tool calls which you don’t review. A constrained language is useful in both cases to guide the model.
5. This Earlier discussion can provide useful background for why this is needed. You have probably seen it, but the readers will likely appreciate. https://news.ycombinator.com/item?id=48051562
The problem is real, Thank you for taking a stab and sharing your findings.
"How <is> this better than just using any other script language?"
I wanted to make the language something built for a machine to write and a human to approve with a narrow scope and my control of what extends that. What shell commands, if any. What MCP and what tools in that MCP, and so on. If an agent generates a Python script to run unattended every morning, I feel I can't control it.
"Inventing a new language complicates large models ability to generate such scripts..."
I tried to keep it as small as possible and borrow a lot from what I knew models had seen in makefiles, YAML-ish, etc.
That was actually the question I began with. What would a make file look like if I were using it to make a classic markdown skill? Asking that question, sort of spawned the rest of it.
I tested the language against cold agents in both frontier model and local model camps to see where they had difficulty nad made changes as required. The lint approach as well as help topics from the MCP helped a lot.
"AI Showing..."
Yea there is a certain irony there. I freely admit product was written with Claude Code. I wrote the Product Requirements & Engineering Requirements and reviewed the code, so I can say I own it.
"Consider the recursive nature of the problem you’re solving..."
That framing is better than mine. When a frontier model writes a skillscript, there's not much chance of something slipping by. Over time, I've become cautiously open to the model adapting the skillscript to solve problems as they occur. Example, a skillscript runs every morning and checks github for PRs and issues. The agent when woken after the run 'noticed' an issue, fixed the script and it was ready for me to approve in the morning. Yes, that could have been done in Python, but I'd not be as confident about it.
I also really appreciate the link.
The problem I'm trying to solve for me is a step removed. The agent is the one writing the script, and I want it to run unattended against my systems every morning. I can't let it write and run bash. Even though my agent lives in a container (NanoClaw), I still need it to reach out to other systems. Github, my other servers, MCPs, etc.
Skillscript is how I let it reach out without holding the keys. It can invoke a skillscript that hits GitHub, but it never runs the raw command or holds the token itself. The runtime holds the credential and only lets it through the skillscript I approved.
It's fair to say a skillscript basically is your two bash scripts and an LLM call, but fenced in.
How can you not get that? Do you believe LLMs remember what you show them between calls? That's not how they work. Each call starts from a clean slate, you have to re-describe the new language each and every call. There's no way to get around that. They are not magic. They do not learn from your prompts, which have absolutely no effect on the model itself.
If you think they do, you are falling for an illusion. ChatGPT is appending each of your incremental prompts to the full prompt, and it grows and grows longer and longer every time you add something. Sure, it summarizes when the full prompt gets to long, but that makes it distort and forget your language definition, and you have to add it again. If you give it the prompt to generate the language from scratch each time instead of the generated language itself, it generates a different language every time. You can't "cleverly hack" or "wish" your way out of that.
They may be good at generating new languages, but one thing that LLMs aren't good at apparently is warning you it's futile to generate a new language intended for llms to program instead of just using existing languages. They just play along and do ridiculous useless things out of syncophancy.
Do you propose just ask AI to generate orchestration in Python?