Contains informed speculation
The choice between footnotes and endnotes (chapter/book) is primarily a stylistic issue and different people have different preferences. You find many discussions on the internet debating the merits of either such as here and here. In my opinion there is no universal standard and different style guides have different recommendations or leave the choice up to the author. NASA prefers footnotes.
I personally prefer footnotes compared to chapter endnotes or book endnotes for ease of reference — I don't have to leave the current page to follow the reference. If footnotes aren't allowed I prefer chapter endnotes. I don't understand the point of book endnotes — why does the author want me to turn all the way to the back of the book to read some parenthetical comment (especially when the book endnotes are separated by chapter)?
You will mostly see footnotes in scholarly papers, especially in the humanities and law. In my experience I have not seen any footnotes or endnotes in scientific papers; they mostly contain references/bibliography (comprising of the earlier sources referred to in the main text) at the end of the paper.
In the case of law books (casebooks, treatises) these do contain footnotes. As for chapter endnotes vs book endnotes it could be a stylistic choice of the author or the house style of the publisher, or if an academic book the style of choice in that field.
As for the discrepancy in books which use chapter endnotes vis-a-vis books which use book endnotes that could be because the Chicago Manual of Style recommends book endnotes over chapter endnotes. The Chicago Manual of Style is also the most widely used in non-journalistic professional writing (according to the Wikipedia).
On the subject of endnote placement the Chicago Manual of Style states (Chapter 14, § 41, hidden behind a 30-day free trial):
Endnotes to each chapter of a book are often best grouped in the end matter, following the text and any appendixes and preceding the bibliography if there is one ... In a book that has a different author for each chapter, or whose chapters may be published separately, endnotes normally appear at the end of each chapter.
And why are the endnotes to each chapter often best grouped in the end? Because that is how books have been traditionally organized. From the Chicago Manual of Style (Chapter 1, § 4):
Books are traditionally organized into three major divisions: the front matter (also called preliminary matter, or prelims), the text, and the back matter (or end matter). ... The text proper comprises the narrative—including arguments, data, illustrations, and so forth—often divided into chapters and other meaningful sections. The back matter presents sources or source notes, appendixes, and other types of documentation supporting the text but outside its central focus or narrative.
And why have books been traditionally organized this way? I have no idea (yet). Maybe because the text and endnotes are usually in different type (size, font) it was easier to segregate them in the olden days when metal plates were used and was therefore cheaper? That would definitely be the case with footnotes.
And why the footnote hate? Once again the Chicago Manual of Style (Chapter 14, § 39):
The limiting factor in printed works is page makeup—it can be difficult or impossible to fit a close succession of long footnotes onto the pages they pertain to, especially in an illustrated work (a basic requirement for all footnotes is that they at least begin on the page on which they are referenced). There is also the matter of appearance; a page consisting almost exclusively of footnotes is daunting.
(The 30-day free trial to the Chicago Manual of Style needs only a working e-mail address)
Example of book with book endnote: Prime Obsession
Example of book with chapter endnote: Against Intellectual Monopoly