Is this really where we've landed? I refuse to believe that any of this markdown insanity will continue indefinitely.
pimlottc 1 days ago [-]
It's insane to me that the "fix" for AI errors is adding more "PLEASE PLEASE DO BETTER" to the prompt
chrismorgan 1 days ago [-]
If it makes you feel any better, the Markdown part is optional (and has no semantics). Somehow it feels about right that the Markdown file can actually just be a YAML file with the wrong extension.
(Actually, to be more specific, a YAML file with no directives, explicitly-signalled start-of-document-content, and followed by a second null document. I will note that frontmatter syntax is not specified; the non-normative Appendix B is the only place that suggests it means prefix and suffix --- lines. And no, frontmatter is not part of Markdown, or CommonMark, and is in fact incompatible with both. And it’s invalid YAML too, the end-of-frontmatter line should be ... to indicate end of document without starting a new document.)
nextaccountic 1 days ago [-]
it's looking like llms are interpreters, and markdown plus english text is the language of choice to run non deterministic programs on it
blooalien 22 hours ago [-]
> it's looking like llms are interpreters, and markdown plus english text is the language of choice to run non deterministic programs on it
That's actually a pretty good clear way of putting it for the typical nerdy "programmer minded" individual.
8cvor6j844qw_d6 1 days ago [-]
QUALITY.md feels similar to CONSTITUTION.md
Looks like unless something better comes up, we'll be stuck with it for a while.
I find markdown useful for repo-specific conventions, especially skills.
formerly_proven 23 hours ago [-]
> I find text useful for repo-specific conventions, especially skills.
stronglikedan 23 hours ago [-]
sure, but structured text like markdown is even more useful, since humans can parse and understand it as easily as skills can
blooalien 22 hours ago [-]
> sure, but structured text like markdown is even more useful, since humans can parse and understand it as easily as skills can
And there are a number of nice viewers / editors either already installed or easily installable on most operating systems to view / edit Markdown in a "beautified" fully rendered form, on both CLI and GUI interfaces; and since most (all these days?) LLMs also "understand" Markdown formatting pretty-much natively, you can easily emphasize certain points to add "weight" to them in the LLMs' "mind" / "thinking" (calculation of statistical token probabilities) process. Plaintext without Markdown is just ... well ... plain. :)
willcodeforfoo 1 days ago [-]
I thought the same about Yaml and Kubernetes/Helm…
vadansky 1 days ago [-]
It already refuses to read the AGENT.md/CLAUDE.md files, what's the point of giving it even more markdown it won't read until you yell at it.
cyanydeez 1 days ago [-]
it is until we define real consistent deterministic gates and protocols. It really is a symptom of the lack of concerted effort. Everyone has a personal preference on how to shove the context and most of them are just "here's some good text I've found to work in my context"
blooalien 22 hours ago [-]
> define real consistent deterministic gates and protocols
I've been experimenting with doing kinda exactly that with the "routing layer" / "harness" level of things, before the "main" LLM itself ever receives the user's input, by getting "user intent" (as a little JSON packet) really quickly from an ultra-lightweight model first and deciding from there in deterministic code what "context" to inject into the user message template, which system prompt to use, and which model to route the assembled context "packet" to for the final response. These LLMs really are fun to play with once you get a feel for which ones do what well, and where each falls short so you can use them each around their individual strengths. :)
dofm 1 days ago [-]
The one thing I do not understand is that here you say:
"Ensure stakeholders are aligned on what matters most and why"
But it is instructions for LLMs, right? A way to describe something that the humans know and the LLMs don't.
LLMs literally cannot be stakeholders, by definition.
craigsmitham 24 hours ago [-]
THe problem is that humans often don't know - this is as much about encouraging getting the humans aligned as the agents. Completely agree agents really aren't stakeholders. Fine point. I'll update description to clarify ... thank you!
chrisweekly 1 days ago [-]
Not OP, but it seems to me the idea is that stakeholders can collaborate and come to consensus on the contents of QUALITY.md.
LiamPowell 24 hours ago [-]
Here's the question I ask about every project that claims to make a LLMs output so much better: If it works so well then why would the model provider not just put it in the system prompt? Or in the case of interactive skills, why would Claude Code/Codex not make it a core part of the product?
On top of that, if your magic markdown file really does work then where's the evidence showing that? These projects never include even basic benchmarks. At best they're entirely vibe based, however more often they're completely untested. Give us a proper benchmark, even a single prompt and it's output with and without your skill in use would be better than every other project out there.
craigsmitham 21 hours ago [-]
No magic. QUALITY.md describes what is unique and valueble to your proejct context that model providers won't have insight into.
istvan0 8 hours ago [-]
You can already put these into the AGENTS.md(or CLAUDE.md) or if it’s too big, you can put it into a SKILL — no need to reinvent the wheel.
hiAndrewQuinn 1 days ago [-]
I'm less interested in this than in what people are willing to aggressively trade off against in order to get the stuff they truly care about.
For example, readability. Where are the developers out there saying "I am very willing to sacrifice a lot of readability to get even a small improvement on e.g. abstraction cleanliness", and sticking with it?
Or "performance can take a huge hit at the cost of being dead easy to read and reason about". Coming up with a list of abstractly good-sounding qualities is just prosocial signaling without knowing what you're willing to sacrifice. There should be a FUCKIT.md that enumerates these.
craigsmitham 24 hours ago [-]
OP here. You're spot on. Trade-offs matter. The trade-offs are implied by the selection of what quality factors/attributes are selected and their requirements. A statement like "performance can take a huge hit at the cost of being dead easy to read and reason about" can sit right there in the QUALITY.md as a comment or in the markdown body.
athrowaway3z 1 days ago [-]
Whats the revenue model for this NBPaaS? (No Bugs Please As A Service)
craigsmitham 21 hours ago [-]
No commercial offering associated with this effort. But a lot of potential for others to incorporate the QUALITY.md standard into products that assess/evaluate quality at varying levels of a loop stack.
The agent skill/CLI that's provided generate a quality evaluation report with recommendations for handoff (ideal for loop engineering) is just one example of how the QUALITY.md file can be used.
It's easy to imagine a SaaS that does the same that provides better eval, reporting, and integration capabilities.
I'd really like feedback on the standard/specification. In short, it defines a quality model of quality factors/characteristics (which you can define as security, reliability, etc), requirements (how those qualities are assessed), a customizable rating scale, and "areas" to have different attributes/requirements for different areas of your project (e.g. frontend/backend, tests, specs, etc). That's basically it - and it follows a consistent pattern of how quality models have been practiced for decades. They are simple and powerful, but - until AI - kind of a pain/toil to get started with. Simple but not easy - until now. QUALITY.md + AI makes it easy. However, you still have to put in the work/care/attention to what goes into your QUALITY.md so you can get maximum leverage from it.
stronglikedan 23 hours ago [-]
So is every proposal to standardize new things, and eventually the cream rises to the top, even though some people are perfectly happy sticking with milk.
Leewen 1 days ago [-]
Useful Nice
formerly_proven 1 days ago [-]
Pure slop.
bellowsgulch 1 days ago [-]
What?
Rendered at 18:02:25 GMT+0000 (Coordinated Universal Time) with Vercel.
(Actually, to be more specific, a YAML file with no directives, explicitly-signalled start-of-document-content, and followed by a second null document. I will note that frontmatter syntax is not specified; the non-normative Appendix B is the only place that suggests it means prefix and suffix --- lines. And no, frontmatter is not part of Markdown, or CommonMark, and is in fact incompatible with both. And it’s invalid YAML too, the end-of-frontmatter line should be ... to indicate end of document without starting a new document.)
That's actually a pretty good clear way of putting it for the typical nerdy "programmer minded" individual.
Looks like unless something better comes up, we'll be stuck with it for a while.
I find markdown useful for repo-specific conventions, especially skills.
And there are a number of nice viewers / editors either already installed or easily installable on most operating systems to view / edit Markdown in a "beautified" fully rendered form, on both CLI and GUI interfaces; and since most (all these days?) LLMs also "understand" Markdown formatting pretty-much natively, you can easily emphasize certain points to add "weight" to them in the LLMs' "mind" / "thinking" (calculation of statistical token probabilities) process. Plaintext without Markdown is just ... well ... plain. :)
I've been experimenting with doing kinda exactly that with the "routing layer" / "harness" level of things, before the "main" LLM itself ever receives the user's input, by getting "user intent" (as a little JSON packet) really quickly from an ultra-lightweight model first and deciding from there in deterministic code what "context" to inject into the user message template, which system prompt to use, and which model to route the assembled context "packet" to for the final response. These LLMs really are fun to play with once you get a feel for which ones do what well, and where each falls short so you can use them each around their individual strengths. :)
"Ensure stakeholders are aligned on what matters most and why"
But it is instructions for LLMs, right? A way to describe something that the humans know and the LLMs don't.
LLMs literally cannot be stakeholders, by definition.
On top of that, if your magic markdown file really does work then where's the evidence showing that? These projects never include even basic benchmarks. At best they're entirely vibe based, however more often they're completely untested. Give us a proper benchmark, even a single prompt and it's output with and without your skill in use would be better than every other project out there.
For example, readability. Where are the developers out there saying "I am very willing to sacrifice a lot of readability to get even a small improvement on e.g. abstraction cleanliness", and sticking with it?
Or "performance can take a huge hit at the cost of being dead easy to read and reason about". Coming up with a list of abstractly good-sounding qualities is just prosocial signaling without knowing what you're willing to sacrifice. There should be a FUCKIT.md that enumerates these.
The agent skill/CLI that's provided generate a quality evaluation report with recommendations for handoff (ideal for loop engineering) is just one example of how the QUALITY.md file can be used.
It's easy to imagine a SaaS that does the same that provides better eval, reporting, and integration capabilities.