Typically the only reason for the lid to be sealed is to prevent the musty odor from the stagnant water from entering the basement. I have not experienced a lid that was caulked shut. Typically they will have nothing or a rubber or a foam type gasket. I would open the lid to see what you have and then figure out if you want to drill a few holes in the lid.

Could it be sealed due to radon? Mine was sealed as part of a radon remediation system.

Do you have the basement plumbed for a future toilet? It may not be a drainage sump at all, but a holding tank that the subteranean toilet will dump into. Systems like this usually have a grinder/pump to lift the sewage up to septic tank level. In this situation, it would be caulked shut and you definitely would not want to drill any holes in it lest you permit it to vent into your basement!

The basement does have plans for a future toilet. However, there is a separate holding pit waiting for a pump that will service the toilet. The sump pump pumps water directly outside, not to a sewer/septic line.

I have not yet looked into this further. Too many other things to do.