Looking for thoughts on my pseudo mutices

While I was trying to produce a platform independent api for mutices & threads that was quick & flexible I ended up with a garbage collecting type api, I figured if I'm going to make use of it for a dedicated garbage collector api (built on top of it) then I should at least find out if anyone sees any potential problems in it, the code is linked below. I'm going to take a break and play a game so don't expect a fast response if you happen to post something.

src/librawpaw/pawmtx.c * 1a302644dbf504d2cc7552eaff45bb96e506b2c5 * Lee Shallis / Dragonbuilder * GitLab

Judging by the lack of responses either peops weren't interested or they didn't see any potential problems, wouldn't be surprised if it's the former, no matter, I'll just have to see in runtime later

Well if anyone WAS interested and didn't see a problem, I did think of a couple before going to bed, 1st was that the mutex at index 0 was the actual pointer handed over during memory allocation, to fix that I just added a header pointer to the top that pointed to the end of the list (the same offset pointer that is handed over to the developer to ensure a crash happens if they try to delete it). The second was the lack of deletion callback for the data that can be associated with the mutices. As for the semaphores that had been #if 0'd out, after doing a bit of research I found I was along the right track since even the linux standard sem_t had a sem_wait call so it's not an issue for me to use mutices under the hood, just need to modify my functions a bit so that the wait can be defined, also change the integer type to unsigned since semaphores aren't supposed to go below 1, well mine permit it but they also have functions that permit reference count modifying so that can be used instead if 0 should not be permitted, currently implementing the fixes to the semaphore section so will pop back later after that and a break to slack off :)

As I was implementing the collector code I realised I had both gone for an awkward api design and made some data race conditions without realising because the code had got way more complicated than intended, I've now re-designed the api a bit and re-wrote the code from scratch, came out cleaner & smaller as a result, anyone lemme know if you see any issues in it (assuming you take a look that is):

src/librawpaw/pawmtx.c * 181019388a78b6342785ecc12820609167dd1af5 * Lee Shallis / Dragonbuilder * GitLab

Found a way to detach mutex pointer protection from a gc backend by using octal permissions, I can't think of any issues I've missed, if you think of any I'd like to know so I can try to address it before I complete enough of the library that I would mark it as releasable.

The file (so you can look up header info):

src/librawpaw/pawmtx.c * cfd388f0eb54e622fe7e1b745738d93198268172 * Lee Shallis / Dragonbuilder * GitLab

The code in said file:

Code:

---

#include "rawpaw.h"

static pawjd owners_perm = PAWIO_O_RW_ACCESS | PAWIO_O_EX_ACCESS;
static pawjd public_perm = PAWIO_O_PUBLIC_RW | PAWIO_O_PUBLIC_EX;
static pawjd groups_perm = PAWIO_O_GROUPS_RW | PAWIO_O_GROUPS_EX;

#ifdef PAW_API_MSW
typedef CRITICAL_SECTION gate_t;
static void open_gate( gate_t *gate ) { LeaveCriticalSection(gate); }
static void lock_gate( gate_t *gate ) { EnterCriticalSection(gate); }
static void term_gate( gate_t *gate ) { DeleteCriticalSection(gate); }
static pawd init_gate( gate_t *gate )
{
        SetLastError(0);
        InitializeCriticalSection(gate);
        return GetLastError() ? ENOMEM : 0;
}
#else
typedef sem_t gate_t;
static void open_gate( gate_t *gate ) { sem_post( gate ); }
static void        lock_gate( gate_t *gate ) { sem_wait( gate ); }
static void term_gate( gate_t *gate ) { sem_destroy( gate ); }
static pawd init_gate( gate_t *gate ) { return sem_init( gate, 0, 1 ); }
#endif

typedef struct _mutex_t mutex_t;

struct _mutex_t
{
        void        *ud;
        pawcls        *cls;
        gate_t *gate;
        paws        name;
        pawd        threadc, allowed;
        pawref        lock, owner, *threads;
};

/* Default callback & class */
static void                void_class_delCB( void *ud ) { (void)ud; }
static pawcls        void_class = { "void", NULL, void_class_delCB };

/* Mutex functions */

RAWPAW_API paws  pawmtx_getname( pawmtx mtx ) { return ((mutex_t*)mtx)->name; }
RAWPAW_API PAWCLS pawmtx_getclass( pawmtx mtx ) { return ((mutex_t*)mtx)->cls; }
RAWPAW_API pawref pawmtx_lockedby( pawmtx mtx ) { return ((mutex_t*)mtx)->lock; }
RAWPAW_API pawjd  pawmtx_getowner( pawmtx mtx ) { return ((mutex_t*)mtx)->owner.tid; }

/* Assumes gate is locked & add is greater than 0 */
static pawref* more_refs( mutex_t *mutex, pawd add )
{
        pawd i = mutex->threadc;
        pawref *refs = pawget( mutex->threads, sizeof(pawref) * (i + add) );
        /* Presumably no memory but never know */
        if ( !refs )
                return NULL;
        mutex->threadc += add;
        mutex->threads = refs;
        pawset( refs + i, 0, sizeof(pawref) * add );
        for ( ; i < add; ++i )
                refs[i].tid = -1;
        return refs;
}

RAWPAW_API pawmtx pawmtx_create( paws name, void *ud, PAWCLS cls, pawd threads )
{
        pawd err = 0;
        gate_t *gate = NULL;
        mutex_t *mutex = NULL;
        cls = cls ? cls : &void_class;
        /* Clear a negative value */
        threads *= (threads >= 0);
        /* Ensure at least 1 slot is allocated for the current thread to be
        * pre-attached */
        threads += (thread < 1);
        if ( !(cls->type && cls->cb) )
        {
                errno = EINVAL;
                return NULL;
        }
        gate = pawgetUD( sizeof(gate_t) + sizeof(mutex_t) );
        if ( !gate )
                return NULL;
        mutex = (mutex_t*)(gate + 1);
        mutex->gate = gate;
        if ( !more_refs( mutex, threads ) )
        {
                pawdel(gate);
                return NULL;
        }
        err = init_gate( &(mutex->gate) );
        if ( err )
        {
                pawdel(mutex->threads);
                pawdel(gate);
                return NULL;
        }
        /* Assign this thread as the owner */
        mutex->owner.tid = tid;
        /* Declare what type of threads are permitted to attach themselves */
        mutex->owner.perm |= 0777;
        /* Declare the number of threads allowed to be just 1, this thread */
        mutex->allowed = 1;
        /* Pre-lock the mutex */
        mutex->lock.ref = 1;
        mutex->lock.tid = pawgetid();
        /* Attach this thread to the mutex */
        mutex->owner.refc = 1;
        mutex->threads->ref = 1;
        mutex->threads->tid = mutex->lock.tid;
        mutex->threads->perm = mutex->owner.perm;
        /* Ensure this points to a valid string even if the string is empty */
        mutex->name = "";
        /* Assign the pointer & data handed to us, do nothing else with it
        * until it's time to delete the mutex */
        mutex->cls = cls;
        mutex->ud = ud;
        mutex->gc = gc;
        return mutex;
}

RAWPAW_API void* pawmtx_attach( pawmtx *mtx, bool withlock, pawjd wait )
{
        pawjd tics = 0;
        mutex_t *mutex = *mtx;
        pawjd tid = pawgetid();
        clock_t prv = 0, now = 0;
        gate_t *gate = NULL;
        pawref *ref = NULL, refs = NULL;
        pawd i = 0, have = 0, timed = (wait >= 0), timeout = 0, gperm = 0, pperm = 0;
        if ( !mutex || !(mutex->valid) )
        {
                pawsig( PAWSIG_SEGV );
                return NULL;
        }
        gate = mutex->gate
        lock_gate( gate );
        /* Check if the current thread has permission to attach itself to the
        * mutex */
        have = mutex->threadc;
        refs = mutex->threads;
        for ( ; i < have; ++i )
        {
                ref = refs + i;
                if ( ref->tid == tid )
                {
                        gperm = ref->perm & mutex.owner.perm & groups_perm;
                        pperm = ref->perm & mutex.owner.perm & public_perm;
                        if ( gperm != groups_perm && pperm != public_perm )
                        {
                                /* Thread permission was revoked, cannot attach until regain
                                * the permission */
                                open_gate(gate);
                                return NULL;
                        }
                        break;
                }
        }
        if ( i == have )
        {
                /* Do we have permission despite not being part of the group? */
                if ( (mutex->owner.perm & public_perm) != public_perm )
                {
                        open_gate(gate);
                        return NULL;
                }
                /* Let's se if we can add the thread to the list of known threads */
                refs = more_refs( mutex, have );
                if ( !refs )
                {
                        open_gate(gate);
                        return NULL;
                }
                ref = refs + i;
                ref->tid = tid;
                ref->ref = 1;
                /* Declare that the thread is not part of the group yet */
                ref->perm = public_perm;
        }
        /* No point looping if don't need to */
        if ( withlock && mutex->valid )
        {
                if ( mutex->tid == tid )
                {
                        withlock = false;
                        mutex->locks++;
                }
                else if ( mutex->tid < 0 && mutex->valid )
                {
                        withlock = false;
                        mutex->lock.tid = tid;
                        mutex->lock.ref = 1;
                }
        }
        ref->ref++;
        /* Total references */
        mutex->owner.refc += mutex->valid;
        open_gate( gate );
        if ( !(mutex->allowed) || !withlock || wait == 0 )
                return mutex->allowed ? mutex->ud : NULL;
        while ( 1 )
        {
                lock_gate( gate );
                /* The pointer may have changed since we released the actual lock */
                ref = mutex->threads + i;
                gperm = ref->perm & mutex.owner.perm & groups_perm;
                pperm = ref->perm & mutex.owner.perm & public_perm;
                if ( gperm != groups_perm && pperm != public_perm )
                {
                        /* Permission to attach was revoked while we were waiting, undo
                        * our increment to the counts */
                        ref->ref--;
                        mutex->owner.ref--;
                        open_gate(gate);
                        return NULL;
                }
                if ( mutex->tid < 0 )
                {
                        mutex->lock.tid = tid;
                        mutex->lock.ref = 1;
                        open_gate(gate);
                        return mutex->ud;
                }
                now = clock();
                tics = now - prv;
                timeout = (tics >= wait);
                if ( timed * timeout )
                {
                        /* Remove our increment before we fail */
                        ref->ref--;
                        mutex->owner.refc--;
                        open_gate(gate);
                        errno = ETIMEDOUT;
                        return NULL;
                }
                open_gate(gate);
        }
        errno = -1;
        return NULL;
}

RAWPAW_API pawd pawmtx_detach( pawmtx *mtx, bool tooklock )
{
        pawd i = 0, have = 0;
        mutex_t *mutex = *mtx;
        pawjd tid = pawgetid();
        pawref *refs = NULL, *ref = NULL;
        gate_t *gate = &(mutex->gc->gate);
        /* Decrement the lock count if the thread had it */
        if ( mutex->lock.tid == tid )
        {
                mutex->lock.ref--;
                /* Release the lock now that the count hit 0 */
                if ( mutex->lock.ref < 1 )
                        mutex->lock.tid = -1;
        }
        lock_gate( gate );
        have = mutex->threadc;
        refs = mutex->threads;
        for ( i = 0; i < have; ++i )
        {
                ref = refs + i;
                if ( ref->tid == tid )
                        break;
        }
        if ( i == have || ref->ref < 1 )
        {
                /* Thread was never attached, dev did something wrong */
                open_gate(gate);
                return EBADR;
        }
        ref->ref--;
        mutex->owner.refc--;
        open_gate( gate );
        return 0;
}
RAWPAW_API pawd pawmtx_revoke( pawmtx mtx, pawjd tid )
{
        pawd i = 0, have = 0;
        mutex_t *mutex = mtx;
        pawjd id = pawgetid();
        gate_t *gate = mutex->gate;
        pawref *refs = NULL, *ref = NULL;
        lock_gate(gate);
        have = mutex->threadc;
        refs = mutex->threads;
        if ( mutex->owner.tid != id )
        {
                /* Only the owner can remove public permissions */
                if ( tid < 0 )
                {
                        open_gate(gate);
                        return EPERM;
                }
                for ( i = 0; i < have; ++i )
                {
                        ref = refs + i;
                        if ( ref->tid == tid )
                        {
                                ref->perm &= ~groups_perm;
                                open_gate(gate);
                                return 0;
                        }
                }
                /* Only the owner can remove group permissions from other threads */
                open_gate(gate);
                return EPERM;
        }
        if ( tid < 0 )
        {
                mutex->owner.perm &= ~public_perm;
                open_gate(gate);
                return 0;
        }
        else if ( tid == id )
        {
                pawd bound = 0, allowed = 0;
                for ( i = 0; i < have; ++i )
                {
                        ref = refs + i;
                        bound += (ref->ref > 0);
                        allowed += (ref->perm != 0);
                }
                if ( bound || allowed > 1 )
                {
                        /* Indicate permission could not be removed because mutex is in use
                        * and doing so would trigger deletion which would crash the app */
                        open_gate(gate);
                        return EADDRINUSE;
                }
                for ( i = 0; i < have; ++i )
                {
                        ref = refs + i;
                        if ( ref->tid == tid )
                        {
                                ref->perm &= ~groups_perm;
                                open_gate(gate);
                                return 0;
                        }
                }
        }
        else
        {
                for ( i = 0; i < have; ++i )
                {
                        ref = refs + i;
                        if ( ref->tid == tid )
                        {
                                ref->perm &= ~groups_perm;
                                open_gate(gate);
                                return 0;
                        }
                }
                /* Thread never got permission to begin with so no removal needed */
                open_gate(gate);
                return 0;
        }
        mutex->owner.tid = -1;
        mutex->owner.perm = 0;
        /* Since only the owning thread can get here and only after all
        * permissions and attachments have been removed it is safe to say the lock
        * is now unused, we set it just in case there's a bug somewhere that
        * allows an unauthorised lock */
        mutex->lock.tid = tid;
        mutex->lock,ref = 1;
        /* We don't want a deadlock happening during the callback so we just let
        * this go in case it does try something stupid */
        open_gate( gate );
        /* Now it's safe to release any pointers */
        mutex->cls->cb( mutex->ud );
        term_gate( mutex->gate );
        pawdel( mutex->gate );
        return 0;
}
RAWPAW_API pawd pawmtx_permit( pawmtx mtx, pawjd tid )
{
        pawd i = 0, have = 0;
        mutex_t *mutex = mtx;
        pawjd id = pawgetid();
        gate_t *gate = mutex->gate;
        pawref *refs = NULL, *ref = NULL;
        lock_gate(gate);
        /* We lock the gate for this check because it is possible that the owner
        * will pass ownership to this thread between our check and rejecting the
        * attempt, locking the gate 1st ensure the change of ownership occurs 1st */
        if ( mutex->owner.tid != id )
        {
                /* Only the owner can add group permissions to threads */
                open_gate(gate);
                return EPERM;
        }
        have = mutex->threadc;
        refs = mutex->threads;
        for ( i = 0; i < have; ++i )
        {
                ref = refs + i;
                if ( ref->tid == tid )
                {
                        /* Doesn't matter if the owner is giving itself group permissions,
                        * it changes nothing as it already had them */
                        ref->perm |= 077;
                        open_gate(gate);
                        return 0;
                }
        }
        refs = more_refs( mutex, have );
        if ( !refs )
        {
                open_gate(gate);
                return errno;
        }
        ref = refs + i;
        ref->tid = tid;
        ref->perm = 077;
        open_gate(gate);
        return 0;
}

RAWPAW_API pawd                pawmkx_passto( pawmtx mtx, pawjd tid )
{
        pawd i = 0, have = 0;
        mutex_t *mutex = mtx;
        pawjd id = pawgetid();
        gate_t *gate = mutex->gate;
        pawref *refs = NULL, *ref = NULL;
        lock_gate(gate);
        if ( mutex->owner.tid != id )
        {
                open_gate(gate);
                return EPERM;
        }
        if ( tid == id )
        {
                /* Why would the owner even trigger this jump? */
                open_gate(gate);
                return 0;
        }
        have = mutex->threadc;
        refs = mutex->threads;
        for ( i = 0; i < have; ++i )
        {
                ref = refs + i;
                if ( ref->tid == tid )
                        goto shift_perm;
        }
        refs = more_refs( mutex, have );
        if ( !refs )
        {
                open_gate(gate);
                return errno;
        }
        ref = refs + i;
        ref->tid = tid;
        shift_perm:
        ref->perm = 0777;
        mutex->owner.tid = tid;
        for ( i = 0; i < have; ++i )
        {
                ref = refs + i;
                if ( ref->tid == id )
                {
                        ref->perm &= ~0700;
                        open_gate(gate);
                        return 0;
                }
        }
        /* Bug if this occurs, indicate as such */
        open_gate(gate);
        return EBADR;
}

---

Seems revolutionary made a brief comment but it got deleted, either by himself or another, guessing himself in light of my more recent comment, anyways I'll respond to this snippet the email showed me

Quote:

---

It's also worth considering how your API will be used in practice. Are there any potential issues or limitations that could arise from using a garbage-collected API for synchronization? How does your API compare to other existing mutex and thread APIs in terms of performance, ease of use, and other factors?

---

I haven't measured performance yet since I'm still cleaning up from my basically thought driven mess that was before. The main problem I had with other implementations is that I did not see anywhere any definitions for what happens when one thread tries to delete the mutex at the same time as another thread trying to lock it, I wanted to make sure that when I delete my shared data objects etc I would not encounter a crash that resulted because some other thread was trying to access the lock, naturally there's a limit to what the mutex can do but I can make it so it doesn't even try to lock the gate if it finds no threads are allowed to bind themselves anyway, I wanted something static to the mutex that had no lock issues besides deadlocks caused by poor usage which is why I used semaphore or critical section, I wanted a way to determine if there where threads allowed to attach themselves, the octal permissions ended up being a simple way to get started and leave potential for further upgrades in a later version of the library, the security bit did concern me a little though since a malicious actor who dug into the details could force their way in so to deal with it I ended up providing the system a way to declare it implemented the mutex & the needed api natively and used a few #ifdefs to map those in by default (KPAWMTX_VER or HPAWMTX_VER for kernel/host respectively), aside from that the GC that I reformed into a linked list that is only intended to used to cleanup at the end of a thread's life, any other time the mutex functions just swap out pointers of linked mutices if they're dying and check if they're at the head via the static head pointer. A thread can call pawmtx_revokeall() any time to revoke all mutex permissions it has on every linked mutex it has, the call is only intended for debug mode however it can work just fine in release mode, it just slows things down a bit since creation & destruction of mutices has to wait for the linked list to be released so they can lock it and do their own business, even the owning thread has to wait, the benefit of the design however is that I can now create thread objects that can be re-attached at any time so long as the permission is still there, as soon as a thread revokes it's permissions, unless the source thread gives it back & notifies it, it not only won't be able to connect to the mutex but would actually be stepping into undefined territory since it's possible the pointer to the mutex it should've abandoned when it revoked it's permissions would no longer be valid anyways, this can be user for interthraed messaging without signals, makes passing data around easier and gives the guarantee that as long as the permissions are not revoked or the thread attachment count is greater than 0 then the pointed mutex & the attached data will not suddenly become invalid, in single threaded situations this is not an issue but in multi-threaded situations it is absolutely an issue, I've also designed the api so that the only way to get the pointer for the data is to call pawmtx_attach() which means it's possible you'll get a NULL pointer and will be forced to abandon whatever you where trying to do. I haven't looked into shared_pointer<> in c++ but I'd wager it's the same purpose just worse since it needs to figure out when the thread is abandoning the pointer altogether, with mine it's possible to just have a single revoke call after passing around a class that uses attach & detach in it's constructor/destructor avoiding the need for it to check if the pointer is still valid at all, it can just assume so instead

This looks pretty useful all things considered.. the only problem I see is that your Makefiles rely on GNU Make exclusive features such as abspath, which is arguably less than desirable for portability concerns.. If you are targeting Windows, Linux, and potentially OS/X in the future, why not go the extra mile and support the BSDs.. although I am not sure of the complexity of the code-base you are trying to port..

I took the liberty of compiling this on my OpenBSD machine. It fails on attempting to find the uchar.h header. I made sure to compile with my local build of GNU Make.

Code:

---

computer# gnu-make-4.3-make
ON_WINDOWS=


===================================================================
cd src/librawpaw && gnu-make-4.3-make --no-print-directory TDIR="${DST_PATH}/prjs" ODIR="${DST_PATH}/objs/librawpaw" -f build.mak all
Project "librawpaw"
CFLAGS=-Wall -Wextra
CFILES=buffer.c encode.c errors.c fileio.c handle.c idbuff.c locale.c locenv.c memory.c module.c paging.c pawls.c pawmtx.c paws.c pawts.c pipefd.c shared.c string.c sysinf.c thread.c timing.c fileio/fio_print.c fileio/iof_print.c fileio/pawfio.c fileio/sysfio.c pawva/printf.c pawva/putnum.c pawva/putstr.c string/pawc.c string/pawhc.c string/pawhhc.c string/pawjc.c string/pawlc.c string/pawllc.c string/pawtc.c
-------------------------------------------------------------------
gnu-make-4.3-make --no-print-directory -f "/tmp/foo/dragonbuilder/paw-shared.mak"
${CC} -shared -Wall -Wextra  -D __linux__ -D __unix__ -D __posix__ -D build_shared_librawpaw -I "${TOP_DIR}/include" -I "${EXT_PTY}/cglm/include" -o "${ODIR}/buffer.c.shared.o" -c "${HERE}/buffer.c"
cc: warning: argument unused during compilation: '-shared' [-Wunused-command-line-argument]
In file included from /tmp/foo/dragonbuilder/src/librawpaw/buffer.c:1:
In file included from /tmp/foo/dragonbuilder/src/librawpaw/rawpaw.h:20:
In file included from /tmp/foo/dragonbuilder/include/paw/raw.h:24:
/tmp/foo/dragonbuilder/include/paw/raw/string.h:13:10: fatal error: 'uchar.h' file not found
#include <uchar.h>
        ^~~~~~~~~
1 error generated.
gnu-make-4.3-make[2]: *** [/tmp/foo/dragonbuilder/paw-shared.mak:21: buffer.c.shared.o] Error 1
gnu-make-4.3-make[1]: *** [build.mak:22: all] Error 2
gnu-make-4.3-make: *** [makefile:184: librawpaw] Error 2
computer#

---

It looks fairly OK all things considered other than that, though.. Aside from that, your Makefiles do not seem to support nmake, or wmake (with nmake belonging to MSVC, and wmake to OpenWatcom respectively), which are both pretty standard Windows compilers all things considered.. You might want to write an nmake file alongside it if you want to also support Windows..

Quote:

---

Originally Posted by interglobber

This looks pretty useful all things considered.. the only problem I see is that your Makefiles rely on GNU Make exclusive features such as abspath, which is arguably less than desirable for portability concerns.. If you are targeting Windows, Linux, and potentially OS/X in the future, why not go the extra mile and support the BSDs.. although I am not sure of the complexity of the code-base you are trying to port..

I took the liberty of compiling this on my OpenBSD machine. It fails on attempting to find the uchar.h header. I made sure to compile with my local build of GNU Make.

Code:

---

computer# gnu-make-4.3-make
ON_WINDOWS=


===================================================================
cd src/librawpaw && gnu-make-4.3-make --no-print-directory TDIR="${DST_PATH}/prjs" ODIR="${DST_PATH}/objs/librawpaw" -f build.mak all
Project "librawpaw"
CFLAGS=-Wall -Wextra
CFILES=buffer.c encode.c errors.c fileio.c handle.c idbuff.c locale.c locenv.c memory.c module.c paging.c pawls.c pawmtx.c paws.c pawts.c pipefd.c shared.c string.c sysinf.c thread.c timing.c fileio/fio_print.c fileio/iof_print.c fileio/pawfio.c fileio/sysfio.c pawva/printf.c pawva/putnum.c pawva/putstr.c string/pawc.c string/pawhc.c string/pawhhc.c string/pawjc.c string/pawlc.c string/pawllc.c string/pawtc.c
-------------------------------------------------------------------
gnu-make-4.3-make --no-print-directory -f "/tmp/foo/dragonbuilder/paw-shared.mak"
${CC} -shared -Wall -Wextra  -D __linux__ -D __unix__ -D __posix__ -D build_shared_librawpaw -I "${TOP_DIR}/include" -I "${EXT_PTY}/cglm/include" -o "${ODIR}/buffer.c.shared.o" -c "${HERE}/buffer.c"
cc: warning: argument unused during compilation: '-shared' [-Wunused-command-line-argument]
In file included from /tmp/foo/dragonbuilder/src/librawpaw/buffer.c:1:
In file included from /tmp/foo/dragonbuilder/src/librawpaw/rawpaw.h:20:
In file included from /tmp/foo/dragonbuilder/include/paw/raw.h:24:
/tmp/foo/dragonbuilder/include/paw/raw/string.h:13:10: fatal error: 'uchar.h' file not found
#include <uchar.h>
        ^~~~~~~~~
1 error generated.
gnu-make-4.3-make[2]: *** [/tmp/foo/dragonbuilder/paw-shared.mak:21: buffer.c.shared.o] Error 1
gnu-make-4.3-make[1]: *** [build.mak:22: all] Error 2
gnu-make-4.3-make: *** [makefile:184: librawpaw] Error 2
computer#

---

It looks fairly OK all things considered other than that, though.. Aside from that, your Makefiles do not seem to support nmake, or wmake (with nmake belonging to MSVC, and wmake to OpenWatcom respectively), which are both pretty standard Windows compilers all things considered.. You might want to write an nmake file alongside it if you want to also support Windows..

---

Thanks, I had planned on splitting via GNUmakefile and makefile names but by the sounds of it I'll need to put more thought than that into that later, then the responding to various make utilities portably. For now I'll focus on the library part of it, making it compile portably and be flexible enough that it get's treated as an attractive option, plus making it compile without the standard c headers, since stdc had to do a lot of compatibility stuff just from their own versions - let alone system stuff - over the years, I want a fresh library that doesn't have all that backward compatibility baggage, I figure I might as well design the library around threads from the get go, these custom mutices are going to the core of that so they need to be rather resilient to data races, can't do anything about devs writing poor/malicious code but I can do something about the data races in the assumed situation that they don't do either (or just fixed their poor code), devs are used to the malloc/free paradigm, I see no reason they won't feel comfortable with the attach/detach paradigm in general code

Quote:

---

Thanks, I had planned on splitting via GNUmakefile and makefile names but by the sounds of it I'll need to put more thought than that into that later, then the responding to various make utilities portably.

---

If you are going to target Linux and Windows as your primary operating systems, and thus want to have multiple Makefiles for the various dialects of make, you might find it helpful to do what I do. Each Make dialect gets its own Makefile, named appropriately. For example, if I wanted to write a program that built with nmake (MSVC) and GNU Make, I might name them as as Makefile.msvc and Makefile.gnu.

Granted, my code targets many different operating systems, so I usually have a slightly different naming scheme from this, but it is definitely a start for you.

Quote:

---

For now I'll focus on the library part of it, making it compile portably and be flexible enough that it get's treated as an attractive option, plus making it compile without the standard c headers, since stdc had to do a lot of compatibility stuff just from their own versions - let alone system stuff - over the years

---

I'll be honest.. disregarding the libc all together is probably not the best idea, and will probably make your code less portable! Glancing at the code base you are working with, you are using stuff from stdarg.h, stdbool.h, stdlib.h, stdio.h, and more. A lot of the stuff found in these headers are (mostly) not functions you want to re-implement yourself!

Overall, the libc is highly consistent across all the operating systems you want to target. You are definitely right in your conclusion that the libc has historical cruft (obviously due to C's legacy), but I would argue that the solution here is to shoot for a standard like C89 (ANSI C) as opposed to C99, which obviously given its older age, will be more feature complete and consistent across all the operating systems you want to target. Granted though, I am looking at your code-base as a whole rather than your standalone library inside of it.

On that note, I want to let you know I have access to many different operating systems I could test your code on. Most of them are virtualized, but I have infrastructure for testing code on them, and I would be more than happy to test your code for you on these platforms if you don't have access to them. The main ones you might be interested in testing on are FreeBSD, OpenBSD, NetBSD, modern Windows, and Linux, although I also have OpenVMS, VAX ULTRIX, UNIXWare, and many others that I would be willing to test on if you wish. Just let me know. :wink:

EDIT(s): Spelling, grammar.

Quote:

---

Originally Posted by interglobber

If you are going to target Linux and Windows as your primary operating systems, and thus want to have multiple Makefiles for the various dialects of make, you might find it helpful to do what I do. Each Make dialect gets its own Makefile, named appropriately. For example, if I wanted to write a program that built with nmake (MSVC) and GNU Make, I might name them as as Makefile.msvc and Makefile.gnu.

---

Roughly what I was planning to do anyways :)

Quote:

---

Originally Posted by interglobber

I'll be honest.. disregarding the libc all together is probably not the best idea, and will probably make your code less portable! Glancing at the code base you are working with, you are using stuff from stdarg.h, stdbool.h, stdlib.h, stdio.h, and more. A lot of the stuff found in these headers are (mostly) not functions you want to re-implement yourself!

---

Only trying to avoid the functions, typedefs and defines are fair game as far as I'm concerned, hence why I used them as the defaults

Quote:

---

Originally Posted by interglobber

Overall, the libc is highly consistent across all the operating systems you want to target. You are definitely right in your conclusion that the libc has historical cruft (obviously due to C's legacy), but I would argue that the solution here is to shoot for a standard like C89 (ANSI C) as opposed to C99, which obviously given its older age, will be more feature complete and consistent across all the operating systems you want to target. Granted though, I am looking at your code-base as a whole rather than your standalone library inside of it.

---

Already shooting for it :) On that note though, I just realised that while I was translating my lua script back into makefiles after realising how to fix my per project settings issues in them I forgot to bring over the -std=c89 -ansi -pendantic options, whoopsie :P I'll have to fix that in a mo, btw do you know what the linux alternative to msw's interlockcompareexchangeacquire function is? Was considering switching to atomics in my mutices if it's a viable option to semaphores and critical sections, I want the mutices to be as fast as possible otherwise there's the strong possibility they would be ignored by most people that do use the library once it's in release state

Quote:

---

Originally Posted by interglobber

On that note, I want to let you know I have access to many different operating systems I could test your code on. Most of them are virtualized, but I have infrastructure for testing code on them, and I would be more than happy to test your code for you on these platforms if you don't have access to them. The main ones you might be interested in testing on are FreeBSD, OpenBSD, NetBSD, modern Windows, and Linux, although I also have OpenVMS, VAX ULTRIX, UNIXWare, and many others that I would be willing to test on if you wish. Just let me know. :wink:

EDIT(s): Spelling, grammar.

---

Nice to know, I'll certainly ask once I've got the library in a state that I would consider stable rather than it's current volatile state, no point checking cross-system runtime if the api isn't even stable due to ongoing research on what each provides.

Quote:

---

btw do you know what the linux alternative to msw's interlockcompareexchangeacquire function is?

---

Off hand, no. I'm not sure if there is one based off of what I'm reading. According to this StackOverflow post, the C11 stdatomic.h library has a family of atomic_exchange functions, where an equivalent can be found here.

Unfortunately, I don't believe POSIX specifies standalone atomic operations like atomic gets, sets, and comparisons. It just guarantees that certain functions in the pthreading library are atomic, but they cannot be broken down to perform more basic operations.

If you are willing to use C11 (MSVC supports stdatomic, at least that is what this blog-post leads to me believe), and so does GCC and Clang (from my own test on my OpebBSD 7.2 machine), so if you are willing to use C11, you don't have much to worry about by the looks of it.

Quote:

---

Originally Posted by interglobber

Off hand, no. I'm not sure if there is one based off of what I'm reading. According to this StackOverflow post, the C11 stdatomic.h library has a family of atomic_exchange functions, where an equivalent can be found here.

Unfortunately, I don't believe POSIX specifies standalone atomic operations like atomic gets, sets, and comparisons. It just guarantees that certain functions in the pthreading library are atomic, but they cannot be broken down to perform more basic operations.

If you are willing to use C11 (MSVC supports stdatomic, at least that is what this blog-post leads to me believe), and so does GCC and Clang (from my own test on my OpebBSD 7.2 machine), so if you are willing to use C11, you don't have much to worry about by the looks of it.

---

Yeah, I don't mind using those when available since they'll be inlined anyways, it's just the functions that don't get inlined that I need to avoid using, basically anything I potentially have to bind libc for is a red flag as far as I'm concerned when it comes to my own library, I'm perfectly happy to learn from the lessons the standard had to learn, I just need to be able to use those lessons in a fresh library built for both modern & ancient development, for instance the standard never included a standard graphics header, I want to learn from that mistake and add one to my own, even if it only provides some basic features common to all libraries like vulkan, as long as it provides those and a means of querying what else is available then that's good enough, devs can just query the handles etc for info they need to start using extensions etc.

Anyways, I now need a sanity check as I've never used the resolution part of time functions so I'm not sure I'm applying it correctly (put aside that I forgot about checking for deadlock situations):

Code:

---

#elif !defined( __STDC_NO_ATOMICS__ )
...
PAWSYN_API pawd pawsyn_time( pawsyn *syn, pawju nanosec )
{
        atomic_long nil = 0;
        struct timespec ts = {0};
        pawju prv = 0, nxt = 0, w8ed = 0, res = 0;
        clock_getres( CLOCK_PROCESS_CPUTIME_ID, &ts );
        res = ts.tv_nsec;
        res *= PAWJ_C(1000000000);
        res += ts.tv_nsec;
        clock_gettime( CLOCK_PROCESS_CPUTIME_ID, &ts );
        prv = ts.tv_nsec;
        prv *= PAWJ_C(1000000000);
        prv += ts.tv_nsec;
        while ( 1 )
        {
                nil = 0;
                atomic_compare_exchange_weak( syn, &nil, 0 );
                if ( nil == 0 )
                        break;
                clock_gettime( CLOCK_PROCESS_CPUTIME_ID, &ts );
                now = ts.tv_nsec;
                now *= PAWJ_C(1000000000);
                now += ts.tv_nsec;
                w8ed += (now - prv) * res;
                if ( w8ed >= wait )
                        return PAWERR_TIME_EXPIRED;
                prv = now;
        }
        return 0;
}
...

---

Can find it here:
include/paw/raw/pawsyn.h * d797d211816ef478eba0da8d7eb4064e2aa62b7d * Lee Shallis / Dragonbuilder * GitLab

Made some improvements to the design so that the mutex could be faster (This upload isn't meant to compile yet so put that aside when making any suggestions), thought some peops might be intested:

include/paw/general/pawmtx.h * dfe0e0aa7371c81e34f269bdd87e2f8a942be748 * Lee Shallis / Dragonbuilder * GitLab