I'm no expert here, but I wonder if a linked list of bounded logs would work well.
So, everyone has a pointer to the current one, and there's an atomic pointer to the next.
When the log fills up, you allocate a new one and set the next pointer to it using compare_and_swap. If someone beat you to it, you can walk the list and add yours to the end so that the allocation isn't wasted.
This way, most of the time you're using the efficient bounded log.
in the case of a bounded log, readers are expected to give an offset at which they want to perfom the read (kafka-like).
So the linked list would involve going through all the links and following end tail refs/pointers. It would make reading O(n) and that's a Nope.
However, you could imagine having a Vec index that contains a ref to all allocated inner-logs, and query the index first in order to obtain the buffers' location. That works, but then the index has to go through a lock (either a RWLock or a mutex) as the CaS operation isn't enough if we get to the end of the index and it needs to be reallocated. It's fine, and I think that's the most appropriate solution.
PS : In fact, there is a sweet spot where you'd like to have a semi-unbounded log. If your index is big enough to contain something like 4Md entries, you'd probably end-up splitting the log in several pieces for archiving and performances purposes. Loading the log (fully or partially) from disk efficiently is then more important than having a real unbounded log. Then you would not necessarily use a lock and could CaS in the index.
So, everyone has a pointer to the current one, and there's an atomic pointer to the next.
When the log fills up, you allocate a new one and set the next pointer to it using compare_and_swap. If someone beat you to it, you can walk the list and add yours to the end so that the allocation isn't wasted.
This way, most of the time you're using the efficient bounded log.