Seminář: Towards an Artificially Intelligent System: Evaluating the Intelligence of an Artificial System

The talk addresses the question how to recognize and evaluate whether an artificial system is intelligent. The question is tightly connected with our understanding and definition of intelligence and as such was present at the beginning of the field of artificial intelligence (AI). However, the original question was ignored as the field developed, and only recently it comes into the research focus again in the area called artificial general intelligence (AGI). In its first part, the talk presents the understanding of intelligence in philosophy and cognitive sciences. Apart from the Strong–Weak, and General–Specific (artificial) intelligence distinctions, it summarizes the approach of the Turing test and its extended versions. The connection with natural intelligence is demonstrated mainly on Psychometric AI. The efforts to define intelligence culminate in the Universal Intelligence definition of Legg and Hutter. The talk follows further development of the definition, especially the Algorithmic Intelligence Quotient (AIQ) test by Legg and Veness. In its second part, the talk describes a replication of Legg and Veness’s experiments with AIQ test on several artificial agents including Freq, Q(0), Q(lambda), H(lambda), and MC AIXI. Three experiments were conducted: default, varying the action space, and varying the observation space. Apart from the results of the experiments, the influence of parameters of MC AIXI on its performance was analyzed, and some more insights on the testing process were acquired. Based on these, new directions of research open.