This feels like a perfect use case for AI.

It might be a good idea to look into the research on streams as coalgebras, there is quite a bit, for example here https://cs.ru.nl/~jrot/CTC20/.

Coalgebras might seem too academic but so were monads at some point and now they are everywhere.

Did consider that your view might be skewed because you work in a CRUD app?

That is very sad to hear.

But you have to know what book you are looking for.

The previews are still there though, they just don't rank.

Right, that's what I'm saying. For whatever reason it seems publishers decided they don't want their preview-only books as part of the full-text search across all books. If they decide that, Google has to comply.

This isn't like web search where web pages are publicly available and so Google can return search results across whatever it wants. For books, it relies on publisher cooperation to both supply book contents for indexing under license and give permissions for preview. If publishers say to turn off search, Google turns off search.

But why would people train on excerpts from Google Books when whole books can be downloaded on libgen and such?

Google books is much bigger than libgen.

copyright reasons?

Both are a copyright violation

The books are still there, it seems like the rankings have changed though.

None of these does full text search.

And they are under constant threat by nation states. sci-hub hasn't seen new papers in ages.

zlibrary does

https://en.wikipedia.org/wiki/Z-Library

Huh, the search is not amazing but it will have to do. Thanks! Are there others?

The Internet Archive supports full-text search on (AFAIK) its entire scanned book collection, even books that aren't available for borrowing.

This is actually pretty good.

I'd wonder if you'd ever consider putting up a downloadable mirror of their full-text search db?

Build a local index

My problem is finding references I don't know about.