AI Center · Czech Technical University in Prague

Viliam Lisý

Associate Professor, Department of Computer Science, FEE CTU

I build AI agents that learn, adapt and reason strategically, both multi-agent reinforcement learning systems and LLM agents that learn in context and remember. I also work on making these agents safe and secure enough for education, cybersecurity and science.

Get in touch Google Scholar LinkedIn CV
Portrait of Viliam Lisý
ScienceDeepStack, the first AI to beat professional players at no-limit poker
~80papers in AIJ, Science, AAAI, IJCAI, AAMAS and more
15+ yrsof research on strategic decision-making under uncertainty
IndustryResearch with Cisco, Trend Micro, Avast and Gen
About

From superhuman poker to agents that learn on the job

I am an Associate Professor at the Artificial Intelligence Center, Faculty of Electrical Engineering, Czech Technical University in Prague, where I lead the Sequential Decision Making Group. My work is about agents that make sequences of decisions with incomplete information, among other agents who may be cooperative, self-interested or adversarial.

I studied theoretical computer science at Charles University and artificial intelligence at Vrije Universiteit Amsterdam, then earned my Ph.D. at CTU. During my doctorate I spent time at Carnegie Mellon University. I was a postdoc in Michael Bowling's group at the University of Alberta, where I co-authored DeepStack (Science, 2017).

I have also worked on industrial problems throughout my career, most recently as Principal AI Scientist and AI Architect at Avast and Gen Digital (Norton, Avast, AVG, Avira, LifeLock) from 2019 to 2024.

Research

Two directions, one question: how should agents learn to act?

Both directions build on 15 years of work in computational game theory. That work gives us principled tools for reasoning about other agents, hidden information and worst-case behaviour.

01

Multi-agent reinforcement learning

imperfect informationsearch + learningself-playmodel-based RLopponent modelling

Scalable algorithms for agents that learn to act among other learning agents, especially when each sees only part of the world. We combine deep RL with look-ahead search and game-theoretic guarantees.

  • Search on top of learning: from DeepStack and continual resolving to look-ahead search over learned abstract models and test-time RL.
  • Model-based MARL: world models for zero-sum imperfect-information games (NashDreamer).
  • Self-play at scale: from DeepStack, which beat professional poker players, to a superhuman agent for the real-time strategy game Generals.io.
  • Adaptation: robust counter-strategies that exploit opponents without becoming exploitable themselves.
  • Foundations: formal models of partially observable multi-agent decision-making.
Multi-agent RL in detail →

Open question: how can agents exploit test-time compute and learned models in strategic settings as effectively as they do in single-agent ones?

02

LLM agents

in-context RLmemory systemssafetysecurity

We take tasks that people do today, automate them with LLM agents, measure where they fall short, and study how to make them better. Our main focus is agents that improve from experience without retraining.

  • In-context reinforcement learning: agents that improve their behaviour from rewards and feedback within their context, across episodes.
  • Memory systems: what an agent should store, retrieve and generalise from past experience to make better sequential decisions.
  • Safety: reliable behaviour of agents that act autonomously over long horizons, and measuring when they can be trusted.
  • Security: agents facing adversaries, such as manipulation, deception and attacks through the environment, analysed through the worst-case lens of game theory.
LLM agents in detail →

Open question: can LLM agents learn from experience as reliably as RL agents, without inheriting new failure modes that adversaries can exploit?

Application domains

Where the agents go to work

Education TAČR

EduGenie: effective and safe use of generative AI tools in the student learning process. The project studies when generative AI helps pupils learn and when it harms learning, including differences by gender, socio-economic background and special educational needs. It will develop and pilot an interactive LLM-based tutor that adapts to individual pupils.

TA ČR SIGMA, TQ23000086 · 2025–2028 · with Scio Research (lead) and the Faculty of Education, Masaryk University

AI in education →

Cybersecurity

Game-theoretic defence and learning-based detection: honeypot and deception strategies, attack-graph games for network hardening, malware detection under concept drift, RL agents for network attack simulation (NASimEmu), and mapping the deception surface of MITRE ATT&CK.

In collaboration with Cisco, Trend Micro, Avast and Gen. Funding from Cisco and the Office of Naval Research Global.

AI and cybersecurity →

AI for scientific discovery

Scientific discovery is sequential decision-making under uncertainty: which hypothesis to test next, which experiment to run, and when to stop. We apply agents that learn in context and keep memory of past experiments to guide the search for new knowledge.

A growing direction. I am looking for domain partners with data, simulators or labs.

AI for science →
Publications

Selected papers

A few key papers. For more, see the multi-agent RL, LLM agents and security pages, or the full list on Google Scholar.

  • 2026
    Decoys Cannot Go Everywhere: Mapping the Deception Surface in MITRE ATT&CK
    Veronica Valeros, Carlos Catania, Viliam Lisý, Harm Griffioen
    arXiv preprint
    arXiv
  • 2026
    Equilibrium Refinements Improve Subgame Solving in Imperfect-Information Games
    Ondřej Kubíček, Viliam Lisý, Tuomas Sandholm
    IJCAI 2026
    DOI
  • 2026
    NashDreamer: Model-Based Reinforcement Learning for Zero-Sum Imperfect-Information Games
    Tomáš Holeček, Viliam Lisý
    arXiv preprint
    arXiv
  • 2026
    Superhuman AI for Generals.io Using Self-Play Reinforcement Learning
    Matěj Straka, Viliam Lisý, Martin Schmid
    arXiv preprint
    arXiv
  • 2026
    Test-time Reinforcement Learning in Imperfect Information Games
    Ondřej Kubíček, Viliam Lisý, Tuomas Sandholm
    arXiv preprint
    arXiv
  • 2026
    Towards Improving Sequential Decision-Making in LLM Agents via Experience Memory
    Jakub Rada, Viliam Lisý
    arXiv preprint
    arXiv
  • 2024
    Look-ahead Search on Top of Policy Networks in Imperfect Information Games
    Ondřej Kubíček, Neil Burch, Viliam Lisý
    IJCAI 2024
    DOIarXiv
  • 2023
    NASimEmu: Network Attack Simulator & Emulator for Training Agents Generalizing to Novel Scenarios
    Jaromír Janisch, Tomáš Pevný, Viliam Lisý
    Workshop at ESORICS 2023 (LNCS)
    DOIarXiv
  • 2023
    Value functions for depth-limited solving in zero-sum imperfect-information games
    Takes DeepStack-style depth-limited search beyond poker to general imperfect-information games.
    Vojtěch Kovařík, Dominik Seitz, Viliam Lisý, Jan Rudolf, Shuo Sun, Karel Ha
    Artificial Intelligence
    DOIarXiv
  • 2021
    Rethinking formal models of partially observable multiagent decision making
    Factored-observation stochastic games: a common formalism connecting game theory and multi-agent RL. Also presented in the IJCAI 2023 journal track.
    Vojtěch Kovařík, Martin Schmid, Neil Burch, Michael Bowling, Viliam Lisý
    Artificial Intelligence
    DOI
  • 2019
    Hardening networks against strategic attackers using attack graph games
    Karel Durkota, Viliam Lisý, Branislav Bošanský, Christopher Kiekintveld, Michal Pěchouček
    Computers & Security
    DOI
  • 2019
    Monte Carlo Continual Resolving for Online Strategy Computation in Imperfect Information Games
    Michal Šustr, Vojtěch Kovařík, Viliam Lisý
    AAMAS 2019
    DOI
  • 2017
    DeepStack: Expert-level artificial intelligence in heads-up no-limit poker
    The first program to defeat professional players in heads-up no-limit Texas hold'em. It introduced continual re-solving with learned value functions, now a standard paradigm for imperfect-information games.
    Matej Moravčík, Martin Schmid, Neil Burch, Viliam Lisý, Dustin Morrill, Nolan Bard, Trevor L. Davis, et al.
    Science
    DOI
  • 2012
    Game Theoretic Model of Strategic Honeypot Selection in Computer Networks
    Radek Píbil, Viliam Lisý, Christopher Kiekintveld, Branislav Bošanský, Michal Pěchouček
    LNCS
    DOI
All publications on Google Scholar →
Teaching

Introduction to Artificial Intelligence

I teach the introductory AI course at FEE CTU together with Branislav Bošanský, in Czech and in English. It covers the foundations of intelligent agents, from search and games to decision-making under uncertainty and reinforcement learning. The lectures are recorded and freely available on YouTube.

Teaching & supervision → Lectures on YouTube (EN)
For consortium partners

Building a European project?

I welcome invitations to Horizon Europe, Digital Europe and bilateral consortia that need strong expertise in agentic AI, multi-agent learning or AI security.

What I can lead

  • Work packages on LLM agents: design, in-context learning, memory, evaluation
  • Multi-agent and adversarial learning, including robust strategies against strategic opponents
  • Safety and security evaluation of agentic systems (red-teaming, worst-case analysis)

What the AI Center brings

  • One of the largest AI research groups in Central Europe, in central Prague
  • Long experience with national, EU, US-funded and industrial projects
  • A team of Ph.D. students and postdocs, and access to compute infrastructure

Good fits

  • Trustworthy and secure AI agents (Cluster 4: Digital)
  • AI for cybersecurity and resilience (Cluster 3, Digital Europe)
  • AI in education and AI for science pilots
For startups & industry

From research to products

I have spent much of my career solving industrial problems. My research grants from Cisco and Trend Micro were on AI for cybersecurity. From 2019 to 2024 I was Principal AI Scientist and then AI Architect at Avast and Gen Digital. There I led the move of a malware detector used by 350 million people from hand-written rules to deep learning, and the AI architecture of an LLM-based scam analysis assistant from concept to release. I also co-authored three US patents.

I know the gap between a promising paper and a system that works for hundreds of millions of users.

I am open to working with startups looking for technical advice, founders looking for a technical co-founder in agentic AI, and companies that want to run a joint applied project (e.g. TA ČR, EU) with the university.

Discuss a project

Typical topics

  • Designing, evaluating and improving LLM agents for real business workflows
  • Agent memory, learning from feedback, and continual improvement in production
  • Safety and security of agentic systems: prompt injection, misuse, adversarial robustness
  • ML for cybersecurity: malware, scam and intrusion detection, concept drift
  • Decision-making under uncertainty, strategic and adversarial settings, RL
  • Technical due diligence and AI strategy for startups and investors
For prospective Ph.D. students & postdocs

Join the group

I am looking for Ph.D. students and postdocs who want to push the limits of AI agents, from multi-agent RL to LLM agents that learn in context. We are very open to new directions that make sense.

Example topics

  • In-context reinforcement learning for LLM agents
  • Memory architectures for agents that learn from experience
  • Safety and security of autonomous agents
  • Test-time search and learning in multi-agent games
  • Agents for tutoring, cyber defence, or scientific discovery

What you get

  • Wide freedom in research directions and flexible working arrangements
  • A solid budget for LLM tokens and access to compute
  • Travel to top conferences, a network of international collaborators, and industry contacts
  • A competitive full-time salary and six weeks of paid vacation, in the historic centre of Prague

What we look for

  • A master's degree in CS or a related field (Ph.D. for postdocs)
  • Strong Python programming
  • Experience with RL, game theory, or LLM APIs and agent frameworks is a plus
  • Curiosity, persistence, and the drive to publish at top venues

Current opening →
Ph.D. at the AI Center →

To apply, email me a short CV, your transcripts, and a paragraph on what you would like to work on and why. Excellent master's students at CTU can also do a thesis in the group.

Let's talk

Email is the fastest way to reach me. Please say whether you are writing about a project, a collaboration, consulting or a position.