Awhile back, I wrote some C# code that interacts with C/C++ code by reading struct layouts directly. This is unsafe because C structs don't have guaranteed layouts unless you use nonstandard pragmas. So I had the C code bypass regular struct {} and lay out data manually by pointer arithmetic, making it consistent and safe. This worked great, until I discovered that booleans are a different size in C and C++. I fixed this too, but only by doing horrible things I'm only 95% certain actually worked
0 Boosts
0 Likes
@slyecho That's not the direction I want the data to go in
@mcc coders, is this what coding is like?
@mcc isn’t this what protobuf and frens are for..?
@beatrix *thinks*
Actually… I think protobuf would be a good fit for our situation. I'm not sure why we didn't go with protobuf. I think maybe it's because I thought our needs were simple and protobuf was complicated, but our needs have grown more complex since this project was started.
I just asked the "client", who says that they've actually surveyed C# implementations of Protobuf and as of last survey they all seem to be very hard to use compared to how easy serialization normally is in C#.
@mcc “C structs don’t have guaranteed layouts”
So how does repr(C) work, then? Does it only work with a specific set of compilers? Is it based on platform conventions?
@bytex64 I may be about to say something horribly wrong, but the thing about repr(C) is that it guarantees you are using the same layout as the platform's standard C layout, but DOES NOT convey any information about what this platform is. This means repr(C) can talk to repr(C) but neither one can totally reliably talk to C# which is using pure pointer manipulation.
P/Invoke *can* talk to repr(C)… unless you fall into the hole with the disaster around boolean sizes.
@dotstdy @bytex64 Okay so on the one hand you are entirely right, but on the other hand: Microsoft, creator of this problem, is both the platform defining body who decides the ABI the C compilers are downstream from, AND the author of the currently most common compiler on the platform, AND the body who both defined and implemented the entire C# language
@ireneista @bytex64 How do you read a struct without knowing its layout?
@mcc @bytex64 you guess. you start by guessing the fields occur in the order they're declared, which turns out to almost always be true.
if you dig into the details of what Rust is doing, it turns out each Rust type has a specific alignment that repr(C) uses, and those are just unchangeable from Rust's perspective. there are additional declarations that appear to change it but they explicitly cannot make the kinds of changes that map to actual variation in C compilers. we ran into this recently with our Vulkan thing.
@mcc @bytex64 traditionally, the solution to this pre-Rust was to design interfaces that were meant to be portable to not rely on structs as anything other than opaque objects that only one side of the FFI or the other actually knew the contents of, and to pass all the important semantic data as function parameters.
@ireneista @bytex64 *points upward* That's what I did in my C# bindings.
I like MsgPack. I wish the CBOR folks had made an improved MsgPack instead of a second incompatible standard
@mcc@mastodon.social @bytex64@awesome.garden @ireneista@adhd.irenes.space
would you really want to use MsgPack (or CBOR) as an in-process ABI though? self-describing formats with variable-length data and no concept of alignment don't seem suited to that at all. maybe I misread what you're saying though
@hatzka @bytex64 @ireneista CBOR/MsgPack was an example ,but, it depends on what the different elements inside the process are?Scripting languages such as Lua, Python and JavaScript are based on self-describing (to the scripting language) dynamic objects with variable length data and no visibility into alignment. So if those languages used something like CBOR or MsgPack internally, that might make interfacing w/ languages like C easier in some ways. We could add alignment if it helps performance
@mcc@mastodon.social @bytex64@awesome.garden @ireneista@adhd.irenes.space
you mean something like Cap'n Proto? I haven't looked at it in a long while but iirc, although it has a schema language, you can use tagged unions and arrays of key-value pairs to get basically this out of it. but making it work as an in-memory representation has a lot of weird tradeoffs.
@ireneista @hatzka i'm just a simple transhumanist from texas and all i want is to be able to perform lookups within a structure without parsing it. and I'm tired of everything being JSON/XML and it being O(n) to ascertain the length of a data packet
I'm thinking about this because I'm having to add to that code right now and it's miserable. I'm not sure why I'm telling you this. I guess I want to stress there are reasons I'm now on team "Nothing but Rust. Nothing but Rust forever. No more C/C++. Rust. What? Okay, Go or Zig are probably fine. Just no C"
@mcc one day someone will invent a C compiler that lets you write C and native Rust together and they will call it Crust
And then... no one will be laughing anymore
@brandonscript this would make me sad
@mcc to be fair if you want layout to be predictable in rust you also need to use C rules...
@mcc The problem is your code won't work on a PDP-10 or a DEC Alpha.
@mcc You could also write a small C program that writes the C# struct definition by using the offsetof of the different fields. Although, I'd just add static_asserts in the C side to check that the the offsets are what you have used in the other one. You may have to keep a couple variations for different platforms, or try to order the fields so that there are no paddings.