Kyriakos G. Vamvoudakis - Publications

Books

[1] G. R. Hudas, D. Mikulski, K. G. Vamvoudakis, F. L. Lewis, E. Gu, Decision and Control for Tactical Behaviors of Autonomous Systems, in preparation, 2014.

[2] D. Vrabie, K. G. Vamvoudakis,  F. L. Lewis, Optimal Adaptive Control and Differential Games by Reinforcement Learning Principles, Control Engineering Series, IET Press, 2012.

Read reviews of Optimal Adaptive Control and Differential Games by Reinforcement Learning Principles written by Warren E. Dixon that appeared in IEEE Control Systems Magazine, vol. 34, no. 3, pp. 80-92, and Journal of Guidance, Control, and Dynamics, vol. 37, no. 3, pp. 1048-1049, 2014.

Patents

[1] K. G. Vamvoudakis, D. Vrabie, F. L. Lewis, Optimal Online Adaptive Controller,” U.S. Patent Disclosure (Pub. No. US 2013/0262353), 2013.

Journals

[1] K. G. Vamvoudakis, F. R. Pour Safaei, J. P. Hespanha, “Output Feedback Event-Triggered Algorithm for Disturbance Attenuation in Unknown Voltage Source Inverters,” in preparation for IEEE Transactions on Power Systems, 2014.

[2] K. G. Vamvoudakis, J. P. Hespanha, “Online Optimal Synchronization of Parallel Islanded Voltage Source Inverters Using Partial Information,” in preparation for IEEE Transactions on Industrial Electronics, 2014.

[3] L. R. Garcia Carrillo, K. G. Vamvoudakis, J. P. Hespanha, “Optimal Adaptive Control for Weakly Coupled Nonlinear Systems: A Neuro-Inspired Approach,” in preparation, 2014.

[4] L. R. Garcia Carrillo, K. G. Vamvoudakis, J. P. Hespanha, “Optimal Tracking for UAVs under Attacks,” in preparation, 2014.

[5] K. G. Vamvoudakis, F. R. Pour Safaei, J. P. Hespanha, “Stochastic Optimal Control for Partially Unknown Markovian Jump Nonlinear Systems,” in preparation, 2014.

[6] K. G. Vamvoudakis, J. P. Hespanha, “Connectivity Maintenance and Flocking in a Networked Team under Jamming Attacks,” in preparation for Automatica, 2014.

[7] K. G. Vamvoudakis, J. P. Hespanha, “Cooperative Q-learning for Rejection of Persistent Adversarial Inputs in Complex Networks,” in preparation for Journal of Machine Learning Research, 2014.

[8] K. G. Vamvoudakis, “Q-learning for Continuous-Time Graphical Games on Large Networks with Completely Unknown System Dynamics,” submitted to IEEE Transactions on Automatic Control, 2014.

[9] K. G. Vamvoudakis, “Non-Zero Sum Nash Q-learning for Unknown Deterministic Continuous-Time Linear Systems,” submitted to Automatica, 2014.

[10] K. G. Vamvoudakis, “Q-learning for Continuous-Time Linear Systems: A Model Free Infinite Horizon Optimal Control Approach,” submitted to IEEE Transactions on Automatic Control, 2014.

[11] Q. Jiao, H. Modares, S. Xu, F. L. Lewis, K. G. Vamvoudakis, “Multi-Agent Zero-Sum Differential Graphical Games for Disturbance Rejection in Cooperative Control,” submitted to Automatica, 2014.

[12] K. G. Vamvoudakis, J. P. Hespanha, “Game-Theory Based Consensus Learning of Double-Integrator Agents in the Presence of Intelligent Attackers,” submitted to IEEE Transactions on Control of Network Systems, 2013.

[13] K. G. Vamvoudakis, F. L. Lewis, W. E. Dixon, “Online Adaptive Learning Solution for Stackelberg Games in Hierarchical Control Problems,” submitted to Automatica, 2013.

[14] M. I. Abouheaf, K. G. Vamvoudakis, F. L. Lewis, “Online Adaptive Learning Solution of Graphical Multi-Agent Discrete-Time Games Using Approximate Dynamic Programming,” submitted to International Journal of Robust and Nonlinear Control, 2013.

[15] K. G. Vamvoudakis, M. F. Miranda, J. P. Hespanha, “Asymptotically-Stable Optimal Adaptive Control Algorithm with Saturating Actuators and Relaxed Persistence of Excitation,” conditionally accepted in IEEE Transactions on Neural Networks and Learning Systems, 2014.

[16] M. I. Abouheaf, F. L. Lewis, K. G. Vamvoudakis, S. Haesaert, R. Babuska, Multi-Agent Discrete-Time Graphical Games and Reinforcement Learning Solutions,” to appear in Automatica, 2014.

[17] K. G. Vamvoudakis, “Event-Triggered Optimal Adaptive Control Algorithm for Continuous-Time Nonlinear Systems,” to appear in Acta Automatica Sinica (Special Issue on Extensions of Reinforcement Learning and Adaptive Control), 2014.

[18] K. G. Vamvoudakis, J. P. Hespanha, B. Sinopoli, Y. Mo, “Detection in Adversarial Environments,” to appear in IEEE Transactions on Automatic Control (Special Issue on Control of Cyber-Physical Systems), 2014.

[19] K. G. Vamvoudakis, D. Vrabie, F. L. Lewis, “Online Learning Algorithm for Optimal Control with Integral Reinforcement Learning,” to appear in International Journal of Robust and Nonlinear Control, 2013.

[20] S. Bhasin, R. Kamalapurkar, M. Johnson, K. G. Vamvoudakis, F. L. Lewis, W. E. Dixon, “A Novel Actor-Critic-Identifier Architecture for Approximate Optimal Control of Uncertain Nonlinear Systems,” Automatica, vol. 49, no. 1, pp. 82-92, 2013. (This paper was No. 18 in the Top 25 Hottest Articles in Automatica, Elsevier, October-December, 2012)

[21] F. L. Lewis, D. Vrabie, K. G. Vamvoudakis, “Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers,” IEEE Control Systems Magazine, vol. 32, no. 6, pp. 76-105, 2012. (This paper was listed in the Top 25 Most Accessed Articles, IEEE Control Systems, 2013)

[22] K. G. Vamvoudakis, F. L. Lewis, G. R. Hudas, “Multi-Agent Differential Graphical Games: Online Adaptive Learning Solution for Synchronization with Optimality,” Automatica, vol. 48, no. 8, pp. 1598-1611, 2012. (This paper was No. 5 in the Top 25 Hottest Articles in Automatica, Elsevier, July-September, 2012)

[23] K. G. Vamvoudakis, F. L. Lewis, “Online Neural Network Solution of Nonlinear Two-Player Zero-Sum Games Using Synchronous Policy Iteration,” International Journal of Robust and Nonlinear Control, vol. 22, no. 13, pp. 1460-1483, 2012.

[24] K. G. Vamvoudakis, M. A. Christodoulou, “Adaptive Control of Mixed-Interlaced forms,” Chaotic Modeling and Simulation Journal: International Journal of Nonlinear Science, July 2012 Issue, pp. 526-542, 2012.

[25] K. G. Vamvoudakis, M. A. Christodoulou, “Adaptive Backstepping Neural Network Control for Mechanical Pumps,” Chaotic Modeling and Simulation Journal: International Journal of Nonlinear Science, January 2012 Issue, pp. 109-122, 2012.

[26] G. Hudas, K. G. Vamvoudakis, D. Mikulski, F. L. Lewis, “Online Adaptive Learning for Team Strategies in Multi-Agent Systems,” The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology (Special Issue on Intelligent Behaviors in Tactical Unmanned Systems), vol. 9, no. 1, pp. 59-69, 2012. (This paper is the Most-Read Article since last year, SAGE Journals, 2012-present)

[27] K. G. Vamvoudakis, D. Vrabie, F. L. Lewis, “Online Learning Algorithm for Zero-Sum Games with Integral Reinforcement Learning,” Journal of Artificial Intelligence and Soft Computing Research, vol. 1, no. 4, pp. 315-332, 2011.

[28] K. G. Vamvoudakis, F. L. Lewis, “Multi-Player Non Zero-Sum Games: Online Adaptive Learning Solution of Coupled Hamilton-Jacobi Equations,” Automatica, vol. 47, no. 8, pp. 1556-1569, 2011. (This paper was No. 17 in the Top 25 Hottest Articles in Automatica, Elsevier, July-September, 2011)

[29] F. L. Lewis, K. G. Vamvoudakis, Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data,” IEEE Transactions on Systems, Man, and Cybernetics, Part B, vol. 41, no. 1, pp. 14-25, 2011. (This paper is the Top Article in Computational Intelligence, BioMedLib, February 2011-present)

[30] K. G. Vamvoudakis, F. L. Lewis, “Online Actor-Critic Algorithm to Solve the Continuous-Time Infinite Horizon Optimal Control Problem,” Automatica, vol. 46, no. 5, pp. 878-888, 2010. (This paper is selected to be included in the Virtual Special Issue of Annual Reviews in Control, as one of the papers with the highest citation rates in the control field. This paper is in the list of the most cited Automatica articles published after 2009 according to Scopus. This paper was No. 23 in the Top 25 Articles in Automatica, Elsevier, April-September 2010)

Chapters and Contributions in Books

[1] K. G. Vamvoudakis, G. R. Hudas, Past, Present and Future of Control Theory Applied to Autonomous Vehicles/Agents in Severe Conditions,” to appear in Advanced Autonomous Vehicle Design for Severe Environments, eds. V. Vantsevich, M. Blundell, Chapter 1, NATO ASI Series, IOS Press, Amsterdam, 2015.

[2] K. G. Vamvoudakis, G. R. Hudas, Terrorist Threats, Agile Vehicle Trajectory Deviation and Critical Reposition in Interaction with Environment,” to appear in Advanced Autonomous Vehicle Design for Severe Environments, eds. V. Vantsevich, M. Blundell, Chapter 2, NATO ASI Series, IOS Press, Amsterdam, 2015.

[3] K. G. Vamvoudakis, G. R. Hudas, New Methods and Techniques in On-line Control and Learning,” to appear in Advanced Autonomous Vehicle Design for Severe Environments, eds. V. Vantsevich, M. Blundell, Chapter 7, NATO ASI Series, IOS Press, Amsterdam, 2015.

[4] K. G. Vamvoudakis, F. L. Lewis, D. Vrabie, Reinforcement Learning with Applications in Automation Decision and Feedback Control,” to appear in Handbook on Computational Intelligence, ed. P. Angelov, World Scientific, 2015.

[5] K. G. Vamvoudakis,  F. L. Lewis, Shuzhi Sam Ge, Neural Networks in Feedback Control Systems,” to appear in Mechanical Engineers’ Handbook, Instrumentation, Systems, Controls, and MEMS, ed. Myer Kutz, John Willey, NY, 2014.

[6] F. L. Lewis, K. G. Vamvoudakis, Neural Control and Approximate Dynamic Programming,” to appear in Encyclopedia of Systems and Control, eds. Tariq Samad, John Baillieul, Springer-Verlag, Berlin, 2014.

[7] K. G. Vamvoudakis, J. P. Hespanha, R. A. Kemmerer, G. Vigna, “Formulating Cyber-Security as Convex Optimization Problems,” in Control of Cyber-Physical Systems, Lecture Notes in Control and Information Sciences, ed. Danielle Tarraf, Volume 449, pp. 85-100, Springer-Verlag, Berlin, 2013.

[8] K. G. Vamvoudakis,  F. L. Lewis, Online Learning Algorithms for Optimal Control and Dynamic Games,” in Reinforcement Learning and Approximate Dynamic Programming for Feedback Control, eds. F. L. Lewis, D. Liu, Chapter 16, IEEE Press Computational Intelligence Series, 2013.

[9] S. Bhasin, R. Kamalapurkar, M. Johnson, K. G. Vamvoudakis,  F. L. Lewis, W. Dixon,An Actor-Critic-Identifier Architecture for Adaptive Approximate Optimal Control,” in Reinforcement Learning and Approximate Dynamic Programming for Feedback Control, eds. F. L. Lewis, D. Liu, Chapter 12, IEEE Press Computational Intelligence Series, 2013.

[10] K. G. Vamvoudakis,  F. L. Lewis, Online Adaptive Learning Solution of Multi-Agent Differential Graphical Games,” in Frontiers in Advanced Control Systems, ed. Ginalber Luiz Serra, Chapter 2, INTECH, 2012.

[11] K. G. Vamvoudakis,  F. L. Lewis, Online Gaming: Real Time Solution of Nonlinear Two-Player Zero-Sum Games Using Synchronous Policy Iteration,” in Advances in Reinforcement Learning, ed. Abdelhamid Mellouk, Chapter 18, INTECH, 2011. (This book chapter has been accessed more than 3000 times, INTECH, 2014)

[12] K. G. Vamvoudakis,  F. L. Lewis, Online Synchronous Policy Iteration Method for Optimal Control,” in Recent Advances in Intelligent Control Systems, ed. Wen Yu, Chapter 14, Springer-Verlag, Berlin, 2009.

Conferences

[1] M. F. Miranda, K. G. Vamvoudakis, J. P. Hespanha, “Optimal Auto-Tuning of PID Gains for Tracking in a Special Class of Second-Order Linear Systems,” in preparation, 2014.

[2] K. G. Vamvoudakis, P. J. Antsaklis, W. E. Dixon, J. P. Hespanha, F. L. Lewis, H. Modares, B. Kiumarsi, “Autonomy and Machine Intelligence in Complex Systems: A Tutorial,” submitted to American Control Conference, Chicago, IL, 2015. (invited 18 pages lead tutorial paper)

[3] Q. Jiao, H. Modares, S. Xu, F. L. Lewis, K. G. Vamvoudakis, “Disturbance Rejection of Multi-agent Systems: A Reinforcement Learning Differential Game Approach,” submitted to American Control Conference, Chicago, IL, 2015.

[4] K. G. Vamvoudakis, J. P. Hespanha, “Online Optimal Switching of Single Phase DC/AC Inverters Using Partial Information,” Proc. American Control Conference, pp. 2624-2630, Portland, OR, 2014.

[5] K. G. Vamvoudakis, “An Online Actor/Critic Algorithm for Event-Triggered Optimal Control of Continuous-Time Nonlinear Systems,” Proc. American Control Conference, pp. 1-6, Portland, OR, 2014.

[6] K. G. Vamvoudakis, “Distributed Games for Multi-Agent Systems,” Proc. Agile Ground Vehicle Dynamics, Energy Efficiency, and Performance in Severe Environments: International Engineering Symposium, Birmingham, AL, 2013. (keynote invited speaker paper)

[7] M. I. Abouheaf, F. L. Lewis, S. Haesaert, R. Babuska, K. G. Vamvoudakis, “Multi-Agent Discrete-Time Graphical Games: Interactive Nash Equilibrium and Value Iteration Solution,” Proc. American Control Conference, pp. 4195-4201, Washington, DC, 2013.

[8] K. G. Vamvoudakis, L. R. Garcia Carrillo, J. P. Hespanha, “Learning Consensus in Adversarial Environments,” Proc. SPIE Defense, Security and Sensing, Baltimore, MD, 2013. (invited paper)

[9] K. G. Vamvoudakis, J. P. Hespanha, R. A. Kemmerer, G. Vigna, “Formulating Cyber-Security as Convex Optimization Problems,” Proc. Workshop on Control of Cyber-Physical Systems, Johns Hopkins University, 2013.

[10] K. G. Vamvoudakis, J. P. Hespanha, B. Sinopoli, Y. Mo, “Adversarial Detection as a Zero-Sum Game,” Proc. 51st IEEE Conference on Decision and Control, pp. 7133-7138, Maui, HI, 2012. (invited paper)

[11] K. G. Vamvoudakis, F. L. Lewis, M. Johnson, W. E. Dixon, “Online Learning Algorithm for Stackelberg Games in Problems with Hierarchy,” Proc. 51st IEEE Conference on Decision and Control, pp. 1883-1889, Maui, HI, 2012. (invited paper)

[12] K. G. Vamvoudakis, F. L. Lewis, “An Online Integral Reinforcement Learning Algorithm to Solve N-Player Nash Games,” Proc. IEEE Multi-Conference on Systems and Control, pp. 697-702, Dubrovnik, Croatia, 2012.

[13] K. G. Vamvoudakis, D. Vrabie, F. L. Lewis, “Adaptive Optimal Control Algorithm for Zero-Sum Nash Games with Integral Reinforcement Learning,” Proc. AIAA Guidance, Navigation, and Control Conference, Minneapolis, MN, 2012. (invited paper)

[14] K. G. Vamvoudakis, F. L. Lewis, “Policy Iteration Algorithm for Distributed Networks and Graphical Games,” Proc. 50th IEEE Conference on Decision and Control, pp. 128-135, Orlando, FL, 2011. (invited paper)

[15] K. G. Vamvoudakis, F. L. Lewis, “Non-Zero Sum Games: Online Learning Solution of Coupled Hamilton-Jacobi and Coupled Riccati Equations,” Proc. IEEE Multi-Conference on Systems and Control, pp. 171-178, Denver, CO, 2011. (invited paper)

[16] K. G. Vamvoudakis, F. L. Lewis, “Multi-Agent Differential Graphical Games,” Proc. 30th Chinese Control Conference, pp. 4932–4939, China, 2011.

[17] K. G. Vamvoudakis, D. Vrabie, F. L. Lewis, Online Learning of Optimal Control Solutions Using Integral Reinforcement Learning and Neural Networks, Proc. 15th Yale Workshop on Adaptive and Learning Systems, Yale University, 2011. (invited paper)

[18] K. G. Vamvoudakis, D.Vrabie, F. L. Lewis, “Online Adaptive Learning of Optimal Control Solutions Using Integral Reinforcement Learning,” Proc. IEEE Symp. Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pp. 250-257, France, 2011. (invited paper)

[19] K. G. Vamvoudakis, M. A. Christodoulou, “Adaptive Backstepping Neural Network Control for Mechanical Pumps,” Proc. 4th Chaotic Modeling and Simulation International Conference (CHAOS), pp. 589-602, Crete, Greece, 2011.

[20] K. G. Vamvoudakis, M. A. Christodoulou, “Adaptive Control of Mixed-Interlaced forms,” Proc. 4th Chaotic Modeling and Simulation International Conference (CHAOS), pp. 603-616, Crete, Greece, 2011.

[21] K. G. Vamvoudakis, F. L. Lewis, “Online Solution of Nonlinear Two-Player Zero-Sum Games Using Synchronous Policy Iteration,” Proc. 49th IEEE Conference on Decision and Control, pp. 3040-3047, Atlanta, GA, 2010. (invited paper)

[22] K. G. Vamvoudakis, D. G. Mikulski, G. R. Hudas, F. L. Lewis, E. Y. Gu, “Distributed Games for Multi-Agent Systems: Games on Communication Graphs,” Proc. 27th Army Science Conference, Orlando, FL, 2010. (This paper won the best paper award.)

[23] G. Hudas, F. L. Lewis, K. G. Vamvoudakis, “Online Gaming for Learning Optimal Team Strategies in Real Time,” Proc. SPIE Defense, Security and Sensing, vol. 7692, 76920W, 2010. (invited paper)

[24] F. L. Lewis, K. G. Vamvoudakis, Optimal Adaptive Control for Unknown Systems Using Output Feedback by Reinforcement Learning Methods,” Proc. 8th IEEE International Conference on Control & Automation, pp. 2138 - 2145, Xiamen, China, 2010.

[25] K. G. Vamvoudakis, D.Vrabie, F. L. Lewis, Online Policy Iteration Based Algorithms to Solve the Continuous-Time Infinite Horizon Optimal Control Problem,” Proc. IEEE Symp. Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pp. 36-41, Nashville, TN, 2009.

[26] K. G. Vamvoudakis,  F. L. Lewis, Online Actor Critic Algorithm to Solve the Continuous-Time Infinite Horizon Optimal Control Problem,” Proc. International Joint Conference on Neural Networks (IJCNN), pp. 3180-3187, Atlanta, GA, 2009. (invited paper and the most cited paper in IJCNN, Arnetminer, 2009-present)

[27] D.Vrabie, K. G. Vamvoudakis, F. L. Lewis, Adaptive Optimal Controllers Based on Generalized policy iteration in a Continuous-Time Framework,” Proc. IEEE Mediterranean Conference on Control and Automation, pp. 1402-1409, Thessaloniki, Greece, 2009. (invited paper)

[28] K. G. Vamvoudakis, M. A. Christodoulou, “Adaptive Backstepping Control for MAPK Cascade Models Using RBF Neural Networks,” Proc. IEEE Mediterranean Conference on Control and Automation, Athens, Greece, 2007.

[29] K. G. Vamvoudakis, M. A. Christodoulou, “Backstepping, Interlaced and Mixed Interlaced Adaptive Nonlinear Control for Biological Models,” Proc. Intelligent Systems and Computing: Theory and Applications Conference (ISYC 06), pp. 196-219, Ayia Napa, Cyprus, 2006.

Book Reviews

[1] K. G. Vamvoudakis, Optimal and Robust Scheduling for Networked Control Systems by Stefano Longo, Tingli Su, Guido Herrmann, and Phil Barber,” to appear in IEEE Control Systems Magazine, 2014.

Tutorials and Workshops

[1] K. G. Vamvoudakis, J. P. Hespanha, Autonomy and Machine Intelligence in Complex Systems, American Control Conference, Chicago, IL, 2015. (invited speakers for the tutorial are P. J. Antsaklis, W. E. Dixon, and F. L. Lewis)

[2] F. L. Lewis, K. G. Vamvoudakis, D. Vrabie, Optimal Control and Online Game Solutions Using Approximate Dynamic Programming, 50th IEEE Conference on Decision and Control (Workshop on Optimal Adaptive Control: Online Solutions for Optimal Feedback Control and Differential Games Using Reinforcement Learning), Orlando, FL, 2011.

[3] F. L. Lewis, K. G. Vamvoudakis, D. Vrabie, ADP for Control of Continuous-Time Dynamical Systems, Multi-Player Differential Games, and Games on Graphs, IEEE Symposium Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), Paris, France, 2011.

Technical Reports

[1] N. Stockman, K. G. Vamvoudakis, L. Devendorf, T. Höllerer, R. Kemmerer, J. P. Hespanha, “A Mission-Centric Visualization Tool for Cybersecurity Situation Awareness,” University of California, Santa Barbara, August 2012.

[2] K. G. Vamvoudakis, J. P. Hespanha, “Optimal Attacks for the iCTF Game,” University of California, Santa Barbara, July 2012.

Theses

[1] Doctoral Thesis: K. G. Vamvoudakis, Online Learning Algorithms for Differential Dynamic Games and Optimal Control, University of Texas at Arlington, May 2011.

[2] Diploma Thesis: K. G. Vamvoudakis, Adaptive Control for MAPK Cascade Models Using RBF Neural Networks, Technical University of Crete, June 2006.

My Erdös number

An upper bound on my Erdös number is 4 (one of the paths is given below).

[1] K. G. Vamvoudakis, J. P. Hespanha, B. Sinopoli, Y. Mo, “Adversarial Detection as a Zero-Sum Game,” Proc. 51st IEEE Conference on Decision and Control, pp. 7133-7138, 2012.

[2] J. P. Hespanha, M. Prandini, S. Sastry, “Probabilistic Pursuit-Evasion Games: A One-Step Nash Approach,” Proc. 39th IEEE Conference on Decision and Control, pp. 2272-2277, 2000.

[3] P. Chen, K. Johansson, P. Balister, B. Bollobàs, S. Sastry, “Multipath Routing Metrics for Reliable Wireless Mesh Routing Topologies,” Computing Research Repository, 2011.

[4] B. Bollobàs, P. Erdös, “Graphs of Extremal Weights,” Ars Combinatoria, vol. 50, 1998.