NHacker Next
  • new
  • past
  • show
  • ask
  • show
  • jobs
  • submit
Lib0xc: A set of C standard library-adjacent APIs for safer systems programming (github.com)
EPWN3D 2 days ago [-]
Author here, I posted this in Show HN but someone clearly beat me to it. So I'll repost my blurb from there.

Various patterns for safer C programming have been cargo-culting around the industry for decades. Because the language evolves intentionally slowly, these patterns rarely get folded into the language as first-class constructs and are passed down through the generations in a sort of oral tradition of programming.

lib0xc leverages GNUC extensions and C11 features to codify safer C practices and patterns into real APIs with real documentation and real testing. Reduce your casts to and from `void *` with the `context_t` tagged pointer type. Enable type-checked, deferred function invocation with `call_t`. Interrogate structure descriptors with `struct_field_t`. Stop ignoring `-Wint-conversion` and praying you won't regret it when you assign a signed integer to an unsigned integer and use `__cast_signed_unsigned`. These are just a few of lib0xc's standard-library-adjacent offerings.

lib0xc also provides a basic systems programming toolkit that includes logging, unit tests, a buffer object designed to deal with types, a unified Mach-O and ELF linker set, and more.

Everything in lib0xc works with clang's bounds-safety extensions if they are enabled. Both gcc and clang are supported. Porting to another environment is a relatively trivial effort.

It's not Rust, and it's not type safety, but it's not supposed to be. It's supposed to help you make your existing C codebase significantly safer than it was yesterday.

My employer holds the copyright and has permitted its release under the MIT license.

uecker 1 days ago [-]
Thanks!

Two notes: GCC has its "access" attributes which can give you similar bounds safety as clang.

Please see also my experimental library. https://codeberg.org/uecker/noplate/ While I do not had enough time to polish it yet, I think it provides some very nice interfaces with improve type and bounds safety, and are also rather convenient.

Also I wonder what parts are redundant if you have FORTIFY_SOURCE ?

(And thank you for working in this topic. If you continue, please reach out to us)

EPWN3D 20 hours ago [-]
I'll have to give the access attributes a look, I hadn't heard of them. (My team were sitting back on gcc-12, so not up to speed on the latest.)

I think I had seen noplate before -- looks like you're taking advantage of the anonymous struct compatibility changes in C23? Those are going to open up a lot of possibilities. Regardless I'd love to stay in touch -- by "us" do you mean the working group?

uecker 19 hours ago [-]
In principle, the are supported already with gcc 12 or even earlier, but this is evolving. https://godbolt.org/z/88qv3jjTM

I use c23 features but also vm-types for bounds checking which are older (i need the statement expression extension though): https://godbolt.org/z/T96T89Yhc

yes, with us I mean wg14 (or just me).

valorzard 2 days ago [-]
This might be a dumb question, but using this + clang bounds-safety, whats the difference between this and something like Zig or Odin.

What do you think C would need in order to reach the user experience of those languages?

EPWN3D 2 days ago [-]
> This might be a dumb question, but using this + clang bounds-safety, whats the difference between this and something like Zig or Odin.

I really need to learn more about Zig, but from what I know, there are still worlds of possibilities that a modern, well-designed language offers over something like lib0xc. Zig's ability to evaluate any expression at compile-time is one such example.

But generally, lib0xc gives you bounds-safety everywhere it can. Languages like Zig and Rust give you type-safety to their own degrees, which I think is a superset.

> What do you think C would need in order to reach the user experience of those languages?

Not really having direct user experience, it's hard for me to say. But if I what I can give you is a list of features that would make large parts of lib0xc irrelevant:

1. Protocols/traits

2. Allocating from a caller's stack frame (think, returning the result of `alloca` to the caller)

3. printf format specifiers for stdint.h types and for octet strings

4. Ability to express function parameter lists as structures

5. New sprintf family that returns a value which is always less than or equal to the size passed (no negative values)

Basically, I think that the C standard should be working aggressively to cut down on the use cases for heap allocation and `void *`. And I think that the bounds safety annotations should become first-class language features.

onlyrealcuzzo 1 days ago [-]
> I really need to learn more about Zig, but from what I know, there are still worlds of possibilities that a modern, well-designed language offers over something like lib0xc.

Doesn't Apple have a nice `defer { }` block for cleanup? Did you include that in lib0xc? I didn't see in on your README.

EPWN3D 1 days ago [-]
I think defer has been included in the next round of working group proposals for C2y, but I don't think Apple's clang has it. Maybe it's there as a language extension and I just didn't see it.

What lib0xc has is some cleanup attributes that you can apply to variables to e.g. automatically free a heap allocation or close a file descriptor, at end of scope. Personally, I like variable annotations much more than defer for these uses, but they accomplish the same thing. I've also found that using those attributes inherently pushes your code to make ownership more explicit. I personally stopped being terrified of double-pointers and started using them for ownership transfers, which eliminates a large class of bugs.

kitd 1 days ago [-]
> I've also found that using those attributes inherently pushes your code to make ownership more explicit. I personally stopped being terrified of double-pointers and started using them for ownership transfers, which eliminates a large class of bugs.

This is very interesting. Do you have a practical example?

EPWN3D 20 hours ago [-]
Yeah here's a trivial one.

void *__free p = NULL;

func(&p); // func zeroes p to claim ownership

// end of scope, p is NULL, nothing happens // if func was not called, p is freed

akoboldfrying 1 days ago [-]
In C++ you can implement such a thing using destructors, which are guaranteed to run in reverse order on scope exit even in the presence of exceptions. Alexei Alexandrescu's Scopeguard did this (in the 90s I think, long before C++11). But in standard C, there's no mechanism that this could be attached to (especially if you want to use "C exceptions", a.k.a. setjmp()/longjmp()).

Maybe the compilers they support all have non-standard extensions that allow something like this though?

shakna 1 days ago [-]
Yes, because all compilers support a non-standard defer mechanism, its now being considered for inclusion into standard C. [0]

And that suggested defer standard, is already available from GCC 9 and clang 22.

[0] https://www.open-std.org/JTC1/SC22/WG14/www/docs/n3734.pdf

lyrie 1 days ago [-]
[flagged]
saidnooneever 1 days ago [-]
Why not pick a different language if you want different features? Why does C specifically need to change, if there are already Zig, Rust etc.?

Why Must C be safe, rather than people writing safer code in it or transfering to other languages if they cannot be bothered?

EPWN3D 20 hours ago [-]
C does not need to be completely safe. But it should be safer by default than it currently is. And I think that "it's a low level language, you just need to be smarter" is too often a cover story for bad design decisions.

I don't think that the choices should be "self-driving cars" and "cars with no seatbelts, airbags, or crumple zones" with nothing in between.

jeremie_strand 23 hours ago [-]
[flagged]
anthk 1 days ago [-]
Wouldn't the last case (void *) hurt embedded C development, or retrogaming with direct memory access and pointers?
debo_ 1 days ago [-]
They said "cut down", not "eliminate."
EPWN3D 1 days ago [-]
Not really. You'd still be able to address memory as bytes. The problem with void * is that it strips all bounds information by its nature. Most of the time when you're passing a void * without an associated length (e.g. context pointers, objects that you pinky-swear are of a certain type), it indicates a failure in the language. That's the stuff I think needs to be eliminated.
anthk 1 days ago [-]
Have a look on libre C SDK's for the GBA and read about how some data it's set. Ditto with another set of archs where some simple C89 it's being ported.
up2isomorphism 1 days ago [-]
Because it is C not Zig/Odin. The mental/ecosystem cost to use a new language is way way under-estimated in most cases.
17 hours ago [-]
17 hours ago [-]
eschaton 1 days ago [-]
Glad to see you’re still doing great stuff, and also very glad to see your new employer supports such things, especially compared to our old employer! Part of why I retired around the same time you left was because I wanted to make and share things.
up2isomorphism 1 days ago [-]
Every time I look at how easy for people to use this kind of thing but people tends not to, remind me if so-called "memory safety" is a real concern anyway.
raggi 2 days ago [-]
there are no good reasons we don't do this in the standards themselves, C, C++, and POSIX should all be working on editions that add safer APIs and mark unsafe APIs as deprecated, to start a long term migration. we know how to do this, we've had a lot of success with this. there are real engineering concerns, sure, but they're not reasons to not do it. compilers and library chains can retain support for less safe variants for plenty of time.
AlotOfReading 2 days ago [-]
The reason this wasn't done by the standards committees is that they spent decades refusing to admit there was even a problem they could help fix. And if there was a problem, it was easily avoided by just writing better code. And if writing better code wasn't enough, well it was certainly too expensive to provide as a debug option. And if it wasn't too expensive to provide as a debug option, the implementors should really lead the way first. And on and on.

The C committee at least seems to get it now. The C++ committee still doesn't, led in large part by Bjarne.

uecker 1 days ago [-]
This is a misrepresentation based on a misunderstanding on how standardization works. The C standard committee has long recognized the need for better safety and carefully made it possible so that C could be implemented safely. But the process is that vendors implement something and then come together during standardization so that it is compatible, not that the standardization is the government that prescribes top-down what everybody has to do. Vendors did not bother to provide safer C implementations and safety features (such as bounds checking) did not get much attention from users in the past. So I am happy to see that there is now more interest in safety, because as soon as there solutions we can start putting them into the standard.

(We can do some stuff before this, but this is always a bit of a fight with the vendors, because they do not like it at all if we tell them what to do, especially clang folks)

anthk 21 hours ago [-]
Stop mixing C and C++, tons of people on Unix still hate C++ (Motif a bit less) for being un-Unixy and megacomplex, even more today. Die had Unix and C people created Plan9 and now Go, which is maybe the other succesor to C before Inferno and Limbo, where programming it's more simpler than the whole C and POSIX clusterfux (even Plan9 and 9front itself can be called a "Unix 2.0").

C++ is something else. Heck, it's often far more bound to a Windows domain (and for a while Be/Haiku) than Unix itself by a huge stretch.

elch 20 hours ago [-]
It is probably worth noting that C++, like C/Unix, originated at AT&T Bell Labs and was originally referred to as "C with classes." Classes were implemented using a preprocessor.

https://www.tuhs.org/Archive/Documentation/TechReports/USG_L...

uecker 21 hours ago [-]
I agree, but are you responding to me?
pjmlp 17 hours ago [-]
Despite all security denial attitude, WG21 is doing much better than WG14.

Still looking forward to the day C supports something like std::string, std::string_view, std::span, std:;array.

Which starting with C++26 finally have a standards compliant story about having bounds checks enabled by default.

pseudohadamard 8 hours ago [-]
Yup, and its not just the standards committees. Look at TR 24731 as an example, an absolute no-brainer for security adding (shock, horror!) bounds checking to long-standing trouble-prone APIs that's been around for 20 years, and the response from most compiler writers/library authors has been "lalalalala I'm not listening I'm not listening". Even then it only got as far as it did due to relentless pressure from Microsoft, anyone else and it'd have been rejected outright.

Having said that, some of it may be due to "it's from Microsoft, we can't ever use it". I'm actually surprised not to have seen any anti-MS diatribes in the discussion so far.

sparkie 1 days ago [-]
The C charter has a rule of "no invention".

Anything needs to be demonstrated and used in practice before being included in the standard. The standard is only meant to codify existing practices, not introduce new ideas.

It's up to compiler developers to ship first, standardize later.

thayne 1 days ago [-]
That produces a bit of a chicken and egg probablem for a stdlib overhaul. Compilers and libc implementations don't have a strong reason to implement safer APIs, because if it is non-standard then projects that want to be portable won't use it , but it won't get standardized unless they do add safer APIs.

So the best hope is probably for a third party library that has safet APIs to get popular enough that it becomes a de facto standard.

EPWN3D 1 days ago [-]
I think the real failing is that new language features then must be prototyped by people who have a background in compilers. That's a very small subset of the overall C community.

I don't have any clue how to patch clang's front end. I'm not a language or compiler person. I just want to make stuff better. There needs to be a playground for people like me, and hopefully lib0xc can be that playground.

jstimpfle 23 hours ago [-]
By adding to the language itself, you mostly make stuff worse. The major reason why C is useful is its quite stable syntax and semantics. Language is typically not the area where you want to add code. It's much better (and much easier) to invent function APIs. See how they shake out, if they're good you might get some adoption.
uecker 1 days ago [-]
Well, there is Annex K which is based on a previous Microsoft effort. Almost universally it is considered terrible and few people implemented it.
avadodin 1 days ago [-]
Immediately what I thought of when I saw /microsoft.

Not all of the APIs were brain-dead. They just ignored all previous developments and in the proposal they didn't even remove the C++-related language.

zbentley 2 days ago [-]
There are only two kinds of standards: ones that prioritize stability and backwards compatibility over usefulness and security, and ones nobody uses.
anthk 1 days ago [-]
C and POSIX aren't related to C++ at all.
raggi 16 hours ago [-]
A vast number of C++ programs import C and POSIX headers directly, so the language level distinction you wish to make isn’t all that relevant to the subject matter.
matheusmoreira 2 days ago [-]
Interesting. I'll be studying this later tonight so I can apply it to my C projects. Especially clang's -fbounds-safety.
thayne 1 days ago [-]
Interesting that a project from Microsoft doesn't support MSVC or Windows.
queuebert 1 days ago [-]
I suspect in 20 years Windows will be a Linux distribution with a compatibility layer.
kristianp 1 days ago [-]
People say that kind of thing on HN every now and then. I have no idea why this idea is around, it's a complete fantasy in my opinion. I say this as someone who mostly uses Linux.
sgbeal 24 hours ago [-]
> I have no idea why this idea is around,

To the best of my fallible knowledge, the notion was first popularized via <http://esr.ibiblio.org/?p=8764>.

kristianp 13 hours ago [-]
Thanks for the link, interesting that it was written in 2020. It doesn't take into account the fact that Office has never been ported to Linux. Many versions can be run on Wine of course.
atilimcetin 2 days ago [-]
The title looks very promising. I’ve added this library to my to-do list to take a deeper look at it. Using this standart library within restricted safe subset of C++ can be a strong opponent for Zig (at least for myself).
EPWN3D 2 days ago [-]
Haven't really verified that it works with C++, but I tried my best to guard the stuff I knew would be problematic with #if __cplusplus. Happy to have a PR that makes C++ happier with it.
atilimcetin 2 days ago [-]
Thanks for the reply. Noted :)
aDyslecticCrow 21 hours ago [-]
At the very least, It's a treasure trove of more advanced c design patterns that are worth knowing.

As the readme describes it as basically established industry patterns passed down through word of mouth.

I find It's difficult to find deeper level C programming techniques like these normally.

jabl 2 days ago [-]
Unfortunate naming. I thought this was about https://libxc.gitlab.io/ but there's an extra '0' in the name here.
ww520 1 days ago [-]
This is great. Other things needed for a great C development environment are a standardized build process plus build tools and a standardized packaging system.
FuckButtons 1 days ago [-]
I think most people who are into c that I’ve met quite like header only libraries. Copy paste as a package manager does have its benefits.
17 hours ago [-]
uecker 1 days ago [-]
I mostly just install a -dev package on Linux and I am done.
anthk 21 hours ago [-]
Not even that; OpenBSD and Hyperbola GNU don't split packages between headers and binaries. Install a package, you get both.
avadodin 10 hours ago [-]
That's a great approach but your examples sound like: the 4–4–2 soccer formation —known for its use by Paris Saint–Germain and Clube de Remo.
aDyslecticCrow 21 hours ago [-]
The Elf interface is a standardised packaging system.

I do NOT want a package manager in my c code. I'm perfectly content with cloning a git repo from my cmake script.

And there is plenty to choose from if you don't like one or another build system.

EPWN3D 1 days ago [-]
Thanks! I agree, a better build story for C projects is desperately needed.
skydhash 22 hours ago [-]
What most people forget is that software is meant to be used as part of the system. The rush to adopt packaging tools like npm and cargo prevent standardization of system tools. On debian, installing tools should be as simple as ‘apt install’, but now you have to check what toolchain version you install then download GB of stuff from the internet. And that for each software. Easy deps donwload for devs means maintenance nightmare for admins and users.
lelanthran 1 days ago [-]
Because that's precisely what is needed: an easy way to ship dependency malware like npn, pip, cargo, etc.

Like it or not, having a little bit of friction prevents pulling in packages with thousands of transitive dependencies.

platinumrad 2 days ago [-]
I truly hope something like this catches on. There is so much low hanging fruit in both the C and C++ standard libraries. Spatial memory could be 90% solved in both languages by mandating the use of safe interfaces.
nxobject 2 days ago [-]
I'm curious – is MSFT using this in production, or is this a "20% time" project? I'm not sure MSVC could compile the GNU extensions used.
EPWN3D 2 days ago [-]
Author here. It is not currently in production, but it is part of a project in Azure which will go to production at some point. I'm actually leaving Microsoft next week and fully intend to keep working on it if I can reach an agreement to do so with my new employer.
ofek 23 hours ago [-]
If an agreement can't be reached with your new employer are you certain that folks at Microsoft will continue maintaining this library? I'd like to experiment with it but have trepidation regarding future development.
EPWN3D 20 hours ago [-]
There are people on my team who are interested in stewarding, so I think it'll be covered.
sgbeal 24 hours ago [-]
> I'm not sure MSVC could compile the GNU extensions used.

The supported platforms only list Linux and Mac. Notably missing from that list is any of the BSDs (not counting what may or may not remain of BSD under the Mac hood).

EPWN3D 16 hours ago [-]
I just haven't tried it on a BSD. No reason it wouldn't work though. Might require a couple of fixes here and there, but generally the library sticks to standard C stuff and uncontroversial POSIX (in the POSIX target).
andrefelipeafos 2 days ago [-]
Quick question for those who've tried it — does this play with existing C codebases incrementally, or is it more of a "new project only" situation? The README didn't make that obvious to me.
EPWN3D 2 days ago [-]
It's designed to be incremental. For example, you can do a search for `sprintf` and replace it with `ssprintf`. The function signature is the same. Any instance of printing to a character array just works. Think of the APIs as "the stuff you usually do by hand, but safer".

If you get compiler errors, it means you were printing to a heap-allocated buffer (or a buffer whose bounds you did not know), and you should be propagating bounds and using `snprintf`.

Integer conversion is the same way. If you have something like

int v1; uint64_t v2;

<stuff happens>

v2 = (uint64_t)v1;

Then you can replace it with

v2 = __cast_signed_unsigned(uint64_t, v1);

and you'll get a runtime trap when v1 is a negative value, meaning you can both enable -Wint-conversion and have defined behavior for when the value in a certain integer type is not representable in another.

thayne 1 days ago [-]
Is there anything in here for something like a "slice" or dynamically sized array that carries its length along with it?
Panzerschrek 1 days ago [-]
Just use compiler option -std=c++20 and use std::span. Don't try reinventing it in C.
lelanthran 1 days ago [-]
If someone needs more than C provides, why on earth would they choose C++?

No rational person is going to want to have to deal with 10x the number of foot guns.

Literally anything when moving from C is better than C++.

Panzerschrek 1 days ago [-]
Switching to C++ is relatively easy in an existing codebase. It's in many cases as simple as renaming a file from .c to .cpp. But for writing something from scratch it's better to use Rust.
uecker 1 days ago [-]
Renaming c. to .cpp may work with ancient c89 code, but not with anything remotely modern. But while the code then is technically C++, it is not better. I still prefer C for new projects to any other language, because I value short compilation time and reduced complexity. For me, this translates in higher productivity and more fun. With modern tooling, also most C issues are detected early.
Panzerschrek 20 hours ago [-]
Slightly tweaking C code to allow it working in C++ is still much easier compared to full rewrite in some other language.
uecker 19 hours ago [-]
Slightly tweaking might not always be sufficient. Reengineering my numerical code would certainly a bit of effort. But anyhow, I do not think C++ is better. Recently I removed one (!) file with templates (which someone else added) from one of my project because it doubled compilation times (in a project with 750 other files or so). I do no need slow build times, more complexity, and more footguns.
safercplusplus 23 hours ago [-]
> It's in many cases as simple as renaming a file from .c to .cpp.

That is rather optimistic, but, for example, scpptool has a feature [1] that auto-converts from C to a subset of C that can (hopefully) be compiled with clang++. If the original C source uses C11 extensions, clang++ seems to generally produce warnings rather than compile errors.

> But for writing something from scratch it's better to use Rust.

scpptool attempts to make C++ a more viable option by enforcing a memory and data race safe subset using a similar safety strategy.

[1] https://github.com/duneroadrunner/SaferCPlusPlus-AutoTransla...

cv5005 23 hours ago [-]
std span is not bounds checked.
pjmlp 17 hours ago [-]
It is when enabling the respective compiler switches for hardned standard library.

Even better in C++26.

akoboldfrying 1 days ago [-]
How things change. Imagine Microsoft circa 2000 publishing an MIT-licensed source code library targeting "the competition"'s compilers, and including some light humour ("Embiggen C's Pit of Success") in the docs.
AnaSpelunker 1 days ago [-]
Look, C is so simple, you can implement a compiler in under 10K lines of code on a new platform. And then spend the your life dodging bugs and come up with a zoo of bizarre macros to keep you safe.
xigoi 20 hours ago [-]
You forgot the part where you have to read an 800-page document that costs 250€ to access.
fudgeonastick 1 days ago [-]
This stuff is amongst my favourite type of engineering.

Practical. Useful. Not sexy. (I am only one of those.)

Bravo!

worthless-trash 1 days ago [-]
Hell, I aim to be all of those.
DeathArrow 1 days ago [-]
I thought Microsoft adopted Rust. Are they back pedaling?
EPWN3D 1 days ago [-]
Microsoft supports memory safety. Rust is 100% the direction for new projects. But there are existing C codebases that are unlikely to be entirely rewritten in a memory-safe language for various reasons. Such projects can significantly benefit from incremental improvements in memory safety.
bananaboy 2 days ago [-]
This is very cool!
Panzerschrek 1 days ago [-]
It's just an excuse not to use safe languages. And adopting it isn't that easy - one need to learn how the new library works or even rewrite old working and tested code with it.
EPWN3D 15 hours ago [-]
This is a tragically misguided view. There are tons of code bases that aren't going to be rewritten in safe languages for various reasons, be it political or technical. You may or may not agree with those reasons, and you may or may not like that these code bases are important, but the fact remains that these projects exist. Giving them a toolset to adopt a broad set of bounds-safe behavior can only be a good thing.
aDyslecticCrow 21 hours ago [-]
If i cannot use GCC i cannot compile to half the embedded platforms i work with on a daily basis.

C remains widespread for unique reasons that not many other languages actually quite grasp.

Using C for a destop application should probably stop being done in light of many more languages more suited for the domain.

But there is no replacement for C in hard embedded systems. And there is no replacement for C in the massive domain of legacy c systems.

Panzerschrek 20 hours ago [-]
It's not an excuse to write C forever, only because some embedded platforms have poor support in compilers other than GCC. Embedded developers should do something to force embedded vendors providing something better than a GCC fork or even something more obscure.
aDyslecticCrow 4 hours ago [-]
I really wish i could at the very least force them to use a modern GCC fork. Please no more c99!
xigoi 20 hours ago [-]
Good thing there are safe languages that compile to C, such as Nim.
aDyslecticCrow 19 hours ago [-]
These kind of effort have potential in my eyes. I've seen some others, and been tempted to write some tools of my own of this type.

I personally struggle with often being stuck on c99, not even c11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact
Rendered at 12:33:19 GMT+0000 (Coordinated Universal Time) with Vercel.