I have this character. I am still debating whether I should allow him to read the first line of the King James Version of the Bible, "In the beginning God created the heaven and the earth..." and cut off right there or just summarize what goes on by narration.
Generally speaking, you cannot include in your own fiction an excerpt from published literature without permission. The US copyright office's Fair Use FAQ recommends always getting permission if there is any doubt.
If the work is in public domain, as the text of many classics are, you could insert the whole thing without permission or even attribution.
Note that, as @evilsoup noted in the comments, the King James Bible is protected by a royal copyright within the United Kingdom until 2039, and if you were to publish there you would need to obtain permission from the Cambrige University Press. (The New King James Version, btw, is a far more recent edition and you'd want to contact Thomas Nelson Publishers for permission.)
BTW, unless you're self-publishing, you shouldn't worry about the excerpts you use aside from noting them and being prepared to substitute something else. If you're writing, WRITE, and worry about permission to use your quotations latter.
When you finally find a publisher, their legal department may very well already have permission to use various quotations in their works, and will certainly be far more aware of the precedent and standards for your market than random strangers on the internet ever could be.
The Bible itself is public domain, but particular versions might not be. So the only thing is not to mention the particular version ie. "Scofield KJV".
If you start using the editor's footnotes or personal inserts, then you may run into an issue.