Indeed, if you want anything resembling true security against people with physical
access to the machine, you need to lock-down everything
from BIOS/UEFI onwards, and encrypt your file-systems whilst you're at it if you store anything private.
There should be information on the Internet about locking-down publicly-accessible machines. Search for "kiosk" mode etc.