The App Library contains a series of app collections, which are created automatically based on categories such as Entertainment and Productivity..
These are displayed in blocks of four icons, in mini two by two grids. Each block generally contains three app icons and a folder containing the rest of your apps in that category.
Like a web page, not all of the content is on screen at any one time, depending on how many apps you have installed of course. So you can scroll with three fingers to find more.
You can quickly skip through the categories using the Containers option in the VoiceOver rotor.
However when doing this, the page is not automatically scrolling as it should when you reach the bottom of what is currently displayed.
Same with App Store app when navigating by heading
This very similar behavior happens when trying to navigate by headings in the App Store when trying to navigate from the "Available Apps" heading to the "Recently Updated" heading when there are too many available apps to be updated than will fit on the first screen.
Similar problems are experienced when navigating by headings through the News app when there are more news articles between two headings that will fit on one screen.
These issues have been around for a long time. Fortunately, as you say, there are workarounds but they are just that...workarounds for a buggy UI that doesn't respond as expected.