Yep, and I do this both on my noname e-ink reader and my phone (fb2reader or moon reader apps there). Downloaded audiobooks, whole another beast, are great in Voice app from F-Droid, if you choose this path.

I didn’t find it inconvinient to read on smaller button phones before and the difference between a dedicated healthier device and a modern smartphone mostly escapes me. This obviously excludes PDFs and manga/comics not adapting to your screen size like a basic e-book in epub/fb2 formats, so if you stick to one of those formats - you’d want an A5-paper sized device or more.

Worse problem with some book for me is not a medium or an interface to consume it, but a lack of concentration, interest and/or habit. Life finds a way, and if you got captured by a book, you’d stop to see or care how you eat through it. But for a regular reading routine it’d be great to think of when and where you’d dedicate some time to enjoy literature, so it’d gain a momentum with you.