Hi Jan,
What tech stack are you using for your frontend? I speak to lots of people using next / nuxt type frontends and you can leverage their server functions to handle writing content back to your Directus project instead of them connecting directly back to your Directus API.
This way, when a user provides the up / down vote, you could generate a cookie that associates that upvote with that specific user.