NHacker Next
  • new
  • past
  • show
  • ask
  • show
  • jobs
  • submit
Launch HN: Context.dev (YC S26) – API to get structured data from any website (context.dev)
SOLAR_FIELDS 17 hours ago [-]
If you want to vibe something that gets you 70% of the way to this well funded startup in like 15 minutes just tell your LLM of choice to create a hook or override the web fetching skill to pipe the content through Mozilla’s readability extension that strips the DOM elements out deterministically, leaving only the content. You can then parse it however you want. Can be done entirely client side, in runtime, with a few JavaScript libraries
TheYahiaBakour 17 hours ago [-]
"the first 90% of the code accounts for the first 90% of the development time. The remaining 10% of the code accounts for the other 90% of the development time."

now for my actual thoughts, if that works for anyone in production reliably, then they should at-least try to do that.

many of my customers moved from in-house solutions because the engineering time spent on maintaining something like this alone makes it quite the undertaking, the infra needed to support something like a few hundred TPS w a browser fleet would cost more than a sub to us in the majority of cases

SOLAR_FIELDS 16 hours ago [-]
Conversely, many of my customers abandoned many of their SaaS solutions because they can now hand roll the 10% of the SaaS solution they need in 15 minutes.

That’s not to say all SaaS is going away. But a good hunk of the single simple solution products are going to suffer

bitlad 7 hours ago [-]
Correction: is not going to, is already suffering.
TheYahiaBakour 16 hours ago [-]
touché
try-working 15 hours ago [-]
the remaining 10% accounts for the other 10,000% of the development time
TheYahiaBakour 14 hours ago [-]
i see you too are an experienced dev
arathis 8 hours ago [-]
Sure. The first 70% is a piece of cake. Get the last 5% right.
klntsky 12 hours ago [-]
Readability is far from ideal. check this out: https://github.com/promptware/readweb

It's a custom pipeline I made to address its shortcomings

ra 10 hours ago [-]
Funny I just did that today to replace Jina with Readability. It's surprisingly good.
aadv1k 13 hours ago [-]
Hi! Congrats on the launch, I gained LOTS of great insights from your comments, particualrly the bits about diffrentiation in a crowded market.

I am building something in agentic automation space, currently it's still under development, but would love to know your journey of ideating -> building -> getting the first customer -> iterating -> and presumably getting into YC.

Am still relatively new to this (19 lol) so I got a long way, but would really appreciate any insights :)

Cheers! Wish you luck with Context.

TheYahiaBakour 12 hours ago [-]
i've yapped quite a bit in this thread already about my journey, but happy to provide any (hopefully useful) insights, you can DM me on X, mention this comment please: https://x.com/mynameisyahia
setgree 1 days ago [-]
I like the clarity, tone, and readability of your webpage. Also your FAQ is refreshing

> When Should I talk to sales? > Talk to sales if you need high-volume pricing beyond 2M credits/month, custom rate limits, SSO / SAML, SCIM provisioning, an uptime SLA, annual invoicing, an MSA / DPA, or a dedicated support channel. Reach us at hello@context.dev or through the contact page.

Would that this were the norm everywhere, rather than (say) a sales rep from Datadog scraping my phone number from who knows where to ask about my company's needs after I sign up for a free account on a whim :)

TheYahiaBakour 1 days ago [-]
happy you noticed that, i put quite a bit of love into the ui, i've found engineers care alot about polish & feel of the webapps they use, even for an api product

i'm an engineer by trade, and always hated things like forcing a sales call, or having hidden credit multipliers, i tried to build this with the same ethos i like for my own dependencies (shoutout axiom.co)

twosdai 1 days ago [-]
Are you using residential proxies? How do you handle websites that don't want to be scraped.

EG if I start passing in Linkedin pages what is your expectation of the result that people would see per profile.

EDIT:

Congrats on the launch seriously hard work, just wanting to understand your scraping stance more. I've worked with a lot of tools on this, didn't mean for my initial comment to be adversarial.

TheYahiaBakour 1 days ago [-]
hi, no worries at all, it came off perfectly fine

yes, we use residential proxies + all requests go are js-rendered, we maintain a caching layer which is 95%+ opted into by customers

it's all included in the credit price, great value compared to alternatives, the business model does rely on scale and our margin gets better the more requests we serve (esp infra cost for k8 + browser fleet)

to answer your e.g., yes public linkedin pages will work fine, anything behind a login we don't really support out of the box until we can figure out a safe way to do so, since that's where red lines are drawn

we step in whenever we see our service is hitting a website more than it should, this usually means reaching out to a customer for clarity on why they are not opting into the cache, we have alot of safeguards around fraud/spam and will let someone know if their request pattern looks like they're causing harm

twosdai 21 hours ago [-]
What major edge cases have you found so far for the browser scraping method, its really complex like dealing with Auth, or pages that use pop ups, content blockers . etc...
TheYahiaBakour 20 hours ago [-]
if it needs auth, we don't do any type of logins at the moment, still figuring out how to do it without attracting the wrong type of customer (someone doing shady stuff)

popups / cookie banners are quite easy to get through with enough heuristics for 99%+ of cases

content-blockers are the same thing as the auth scenario, hard to figure out where to draw the line between building something people want and doing something shady

blast 1 days ago [-]
Is "residential proxy" a euphemism for botnets or is there a difference? Genuinely curious.
TheYahiaBakour 20 hours ago [-]
- worst is "extension" based traffic routing where your provider is essentially tricking end users into routing traffic through - opt-in p2p networks can be described as botnets sometimes, i try to avoid these but not as bad - good enough dISP proxies can be classified as residential quality, work quite well, although people have argued w me on their quality - managed business/enterprise residential gateways are ideal, because businesses opt in and allow the traffic - few other options
17 hours ago [-]
dataviz1000 23 hours ago [-]
Have a look at Intercept. [0] I don't have a need for it, likely it is dated and will require some more tuning, and I want to get away from scraping. Creating typed Typescript proxy API for any website might be something you find useful.

> Reverse-engineers any website by doing a breadth search across every transport (JSON, WebSocket, WebRTC, GraphQL, SSE, HLS, PubSub), listing them all, and generating a typed JSON API that bypasses almost all bot protections — including Turnstile. I didn't include the ability, but it bypassed the most advanced ChatGPT + Turnstile. Built with self-improving Claude Code agents that rewrite their own instructions until fresh agents consistently succeed.

> Once connected to a page, it intercepts every byte of network traffic — then actively drives the page to surface endpoints that only fire on interaction. It types into forms, clicks buttons, scrolls, triggers modals, paginates, submits searches, and walks through multi-step flows, watching what each action produces on the wire. Every request gets captured with its method, headers, payload shape, and response, then classified by transport (JSON, WebSocket, WebRTC, GraphQL, SSE, HLS, PubSub). The result is a complete map of the site's real API surface — including the hidden endpoints that only exist behind a click — turned into typed proxy routes you can curl.

[0] https://github.com/adam-s/intercept

TheYahiaBakour 23 hours ago [-]
Fascinating, this is definitely a path i was thinking about for a bit but decided against doing it primarily since it changes the business fundamentally
dataviz1000 23 hours ago [-]
You are smart!

Another YC startup went this path, scraping targeting endpoints, and they didn't get much traction and they have since pivoted.

> We maintain a caching layer and avoid hammering websites.

This made me think of it. It is another way to avoid hammering websites by using extremely targeted requests for the most part bypassing HTML, DOM, and JavaScript.

TheYahiaBakour 22 hours ago [-]
100% yes, i found out about that company recently too, it's a tempting idea for sure
Barbing 19 hours ago [-]
Why not much traction, was the benefit more to websites than businesses/scrapers?
TheYahiaBakour 19 hours ago [-]
it counts as "reverse engineering" a website which is much worse for the website owner imo, then again im biased
xnorswap 5 hours ago [-]
FYI Your docs don't respect system-mode choice until you click into light/dark mode then back to system.

They just show up in dark mode if there's no local storage setting set yet.

TheYahiaBakour 2 hours ago [-]
is that a problem/annoying? i thought it would be fine
xnorswap 48 minutes ago [-]
It's clearly a bug. The UI defaults to "system preference" but then gives you dark mode anyway, even if your system preference is light mode.

To get back to actually using system preference, you have to choose light/dark then choose system preference again.

htrp 2 hours ago [-]
how's this different from offerings from cloudflare, firecrawl, tavily, exa?
jackienotchan 14 hours ago [-]
How did you find your differentiation in a highly commoditized space? It's probably one of the most crowded spaces.

Even within YC, there are many competitors that do pretty much the same thing:

- Firecrawl

- BrowserUse

- Browserbase

- CloudCruise

- NotteLabs

- Intuned

- Expand.ai

- Reworkd

And then you have the extremely well-funded web retrieval players like Parallel and Exa.

How do you differentiate to all these?

Another thing that might interest HN: AI crawlers come with negative side effects for website owners (costs, downtime, etc.), as repeatedly reported here on HN (and experienced myself).

Does Context respect robots.txt directives and do you disclose the identity of your crawlers via user-agent header?

TheYahiaBakour 14 hours ago [-]
the reality is that the space is huge and expanding

many that you mentioned serve different needs and it tends to not be zero sum to everyone's surprise (including mine at first)

e.g.

browseruse, browserbase = browser automation exa, parallel = search for ai (primarily)

we differentiate by prioritizing the infrastructure use-case, we encourage people to build products & features on top of our APIs and build it accordingly

some direct differentiators are: no hidden credit multipliers, world class brand data, and an api that just works

we're trying to be good samaritans on the web, and cache data whenever possible so we can avoid overloading a website, it's the main reason we don't offer "login with ai" features even though it's trivial at this point, it's quite hard to figure out how to avoid shady behavior while building something people want, i don't have a good answer yet and neither do many others

lastly, to reiterate, lots of our customers migrate from alternatives, but lots also just use multiple providers

modo_ 1 days ago [-]
I was using Context back when it was still Brand.dev. I found it to be a great product- one of those rare APIs that immediately made a problem I had disappear. Had it in production within an hour of signing up

Agents need clean/current context from the web, and this is the best way I’ve found to give it to them. The internet is clearly moving in this direction: companies are starting to realize their sites need to be legible to agents. Some are already adapting but many haven’t yet. Context feels like an important part of that transition

Yahia is a great builder. His pace of expansion has been impressive, excited to see where he takes Context.

TheYahiaBakour 1 days ago [-]
oh wow, happy to see you here, we didn't post this anywhere so its nice to see a customer find us so quickly, thank you for the kind words.

brand data is a shockingly hard problem to get right

erichi 1 days ago [-]
[dead]
m_w_ 1 days ago [-]
Unclear what difference exists against Firecrawl - their team has been shipping great features extremely quickly lately, and their core offerings have become really good.

I am interested in KnifeGeek though - looking for a good OTF (ultratech?)

TheYahiaBakour 1 days ago [-]
There's alot of overlap at the moment, but we're diverging quite quickly.

If you want differences as of right now

- 1 credit = 1 scrape, no hidden credit multiplier - we have world class brand data - we're focused primarily on the infra use-case, rather than gtm & everything else - anecdotally, customers have seen their error rates drop quite dramatically

In general it's a huge space, firecrawl is a wonderful company, it's fun to compete with them, planning to add more things soon which should make the differences clear

m_w_ 1 days ago [-]
Interesting - I'll be sure to benchmark it at some point. We've found the best results come from blending providers depending on the task anyways.

Thanks for the quick response - and always happy to see more competition in the space. Best of luck with future features!

TheYahiaBakour 1 days ago [-]
please do, and if you see anything off let me know, we've yet to lose a single "bakeoff", i normally ask customers to just have cc/codex run it so its somewhat unbiased

on the 2nd point, most industries are not zero-sum, and many of customers use multiple data providers in any case, so agree with you there

thank you!

TheYahiaBakour 1 days ago [-]
oh also on knifegeek, it'll be live again this weekend when i find a moment to fix the bug causing it to crash, i collect knives/watches so it was a super fun project to work on
wbnns 9 hours ago [-]
Congrats Yahia, cool to see all the progress and momentum behind what you all are building.
TheYahiaBakour 2 hours ago [-]
thank you!
sheept 1 days ago [-]
Does this respect llms.txt and robots.txt, or have you found it more effective if agents see what humans see?
TheYahiaBakour 1 days ago [-]
we try to read robots.txt, it's definitely more efficient if agents just see what humans see, we run a custom browser rendering stack

in terms of llms.txt, we're not primarily an AI product (although some features do use LLMs), and speaking to friends who run products it seems to be not very helpful, even though we have one as well, i didn't see it move the needle much

even my own coding agents don't look at llms.txt when looking at our own website, so unsure of whether that standard will survive the test of time

hollow-moe 8 hours ago [-]
> Websites can opt out of our service, and we respect these requests and add them to our block list.

I.e: "robots.txt already exists and is commonplace but if we actually followed it our solution wouldn't work so all websites on the planet need to do something specific to opt-out our ddos." lmao

korrectional 24 hours ago [-]
Hey Yahia, I recently saw your startup on the YC registry and found it to be one of the most promising of the batch, so good luck with this! Do you mind if I ask how you managed to get your first users on Brand.dev?
TheYahiaBakour 24 hours ago [-]
hey, happy you like it, to be honest we had no real secret, it takes a ton of time and effort to earn developer trust

i got my first customer after 1 month, and then just worked like hell to make him happy even though it was a very large company paying me $99/mo at the time, they're still a customer too

since then, i just locked in on every customer until they were thrilled, and once we hit 50, word of mouth started to kick in, now we're at 300+ and get a referral every other day which is awesome

i think our community appreciates effort, and when someone goes the extra mile to make sure your problems are solved, you tend to remember that. it doesn't scale super well, but you'd be surprised how far it'll take you

Barbing 19 hours ago [-]
Excellent way to run a business.
lejeanvaljean 7 hours ago [-]
This seems useful, in 2010, when AI could not make this possible in a single prompt. Now I don't see the need for this.
TheYahiaBakour 2 hours ago [-]
i too would have had this view before i started the business, however it's actually really hard, additionally teams migrate to us from in-house solutions all the time
tomerlir 21 hours ago [-]
Really cool idea! I had something similar a while back. Maybe figure out a way to use MCP so the client can connect directly out of the box
TheYahiaBakour 21 hours ago [-]
bwm 1 days ago [-]
Awesome! Been great watching this product improve so quickly, can't wait for what's next :)
TheYahiaBakour 1 days ago [-]
thank you, i try to ship updates often: https://docs.context.dev/changelog
rao-v 12 hours ago [-]
This seems like a responsibly designed service, but I find it a little odd and baffling that we need such intermediaries for the average hobbyist / small project to reliably access sets of content published on the internet.

Wasn’t this the point of the web?

TheYahiaBakour 12 hours ago [-]
i think about this too, perhaps you're right, if (big if) the web was designed with the end goal in place, however it was a series of ~interesting~ mistakes & decisions that led to where we are today

the same could be said of most cybersecurity companies too

debo_ 22 hours ago [-]
Remember when the Semantic Web attempted to make the entire web machine-readable?

https://en.wikipedia.org/wiki/Semantic_Web

TheYahiaBakour 22 hours ago [-]
you just found my end goal :)
kartik_malik 1 days ago [-]
love the design... congrats
TheYahiaBakour 1 days ago [-]
will let my designer know, it's a dev-focused product, and we're all so finely tuned to avoid slop, so design & feel was really important
zuzululu 1 days ago [-]
new frontier models do this already
TheYahiaBakour 1 days ago [-]
im not sure the models do this, rather the tooling around them, however web search/extraction by the model providers is ridiculously expensive and quite slow, so going into production it makes sense to use a provider (like us)
archerx 1 days ago [-]
Great, another thing I have to block server side. Reminds me of the image leech protections that had to be in place because bandwidth was expensive. History doesn’t repeat but rhymes as they say.
pavlov 1 days ago [-]
Maybe AI-service-blocklist-as-a-service could be a YC company.
paytonjjones 1 days ago [-]
Better business model would be some sort of micropayment setup - allow humans, but make scrapers pay a hundredth of a cent for access.
owlninja 1 days ago [-]
paytonjjones 1 days ago [-]
Yep. It will be interesting to see if things move that way or not. AI scraping might get annoying but a lot of sites (eg e-commerce) have obvious incentives to allow AIs to scrape them for free (or even pay AI companies to prioritize them, like they already do with Google).
asdev 1 days ago [-]
So basically web scraping as a service with an API on top?
TheYahiaBakour 1 days ago [-]
yes but we do alot more that may not be clear at first glance, things like brand data, and for scraping handling pdf, ocr, docx, ppt, xlsx automatically

shipping a bunch of new things soon which should make it clearer, but as of today yeah

jubalfh 5 hours ago [-]
fuck you too.
seper8 1 days ago [-]
Seems wildly expensive, furthermore not a single mention of "ip" on homepage? Not using rotating ip's, residential proxies?

AKA unusable for high value data.

TheYahiaBakour 1 days ago [-]
you're welcome to try it on our demo page (no signup needed), should handle everything just fine, yes we don't mention ips on the homepage

also, while it might seem expensive, we're cheaper than every other option out there, because there's no hidden credit multipliers. every single customer who uses us halved their bill + error rate

selfheal 6 hours ago [-]
[flagged]
holydementor 10 hours ago [-]
[flagged]
Kaggbac 5 hours ago [-]
[flagged]
gokulrajaram 17 hours ago [-]
[flagged]
goofrey 7 hours ago [-]
[flagged]
jkwang 9 hours ago [-]
[dead]
ryss20 1 days ago [-]
[flagged]
Fno44 17 hours ago [-]
[flagged]
22 hours ago [-]
laalshaitaan 20 hours ago [-]
lessgooo! context dev ftw!
TheYahiaBakour 20 hours ago [-]
thank you, im not sure who this is though, but please email me if i can help in any way
badmonster 1 days ago [-]
[dead]
growupdorks 22 hours ago [-]
[flagged]
TheYahiaBakour 19 hours ago [-]
ignoring the obvious ragebait, the soc 2 "thing" is the truth, we are type 1 certified, and are in observation for type 2
Barbing 19 hours ago [-]
The year is 2003

You’re on the playground

“That’s so gay”

Age: 12

cahaya 1 days ago [-]
Nice, @grok how does it compare to Cloudflare that also provides a REST endpoint for structured markdown data and screenshots?
TheYahiaBakour 1 days ago [-]
im not sure if grok is on here, i think that's an X thing

but if you were to ask me, we're more fully featured than cloudflare, and anecdotally a ton more reliable in terms of error rates. back when it was brand.dev, i actually tried to use cloudflare's apis and it was quite unreliable, so we built our own stack instead

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact
Rendered at 17:55:27 GMT+0000 (Coordinated Universal Time) with Vercel.