Google lets users try their apps with new features on the Beta stage. The similar thing is happening right now. The Google Messages is now on Open Beta testing, however, due to fast pace signups, the positions are currently full for the testing.
All you have to do is (when available), head out to this link, and click on Become a Beta Tester. After that, you have the update available for download, and the app (if already installed) will be updated.
Before you try out the new updates, do note that the Google Messages Beta might have some bugs to resolve. This can affect your day to day usage of the app. Also, do note that the new app can crash anytime before even you try using it. Do send feedback whenever you face such issues so that the public rollout can be clean of bugs.
Also, if you want to know if the app on Play Store (On PC browser) has Beta program or not. I introduce you to the Chrome Extension by APKMirror & Android Police.
This extension, will let you know if the current app you are checking on Play Store website, has Beta or Not. It can come handy many a times, when you are excited to check new features of an app.
For Chrome: Chrome Web Store
For Firefox: Firefox Add-Ons