Solomonoff induction has led to many useful tools for thinking about inductive inference (including Kolmogorov complexity, the universal prior, and AIXI), but the problem becomes decidedly more difficult in the case where the agent is a subprocess of the universe, computed by the universe.Imagine a black box, with one input chute and two output chutes. A solid understanding of topology turns out to be helpful in many unexpected places.Formalizing beneficial goals does you no good if your smarter-than-human system is unreliable.
Once you understand one topic well, you’ll be ready to try contributing in that topic area on the IAFF.It’s important to have some fluency with elementary mathematical concepts before jumping directly into our active research topics. (It is, of course, still difficult to validate that the specification describes the intended behavior.) The simplest generalization, from this data, might be that humans really like human-shaped smiling-things: this agent may then try to build many tiny animatronic happy-looking people.With all of the material in this guide, please do not grind away for the sake of grinding away. If one of the active research areas fails to capture your interest, switch to a different one. Students who get selected for the internship will deal with the following areas of research: machine learning, deep learning, artificial intelligence, natural language processing, systems biology and health care, smart cities and transportation, financial analytics, reinforcement learning and bandits, and system architectures for data science and artificial intelligence, and artificial intelligence on the edge. Unfortunately, most of them don’t explain their motivations well, and have not yet been put into their greater context.In the case where the agent is embedded inside the environment, the induction problem gets murky: what counts as “learning the universe program”?
What sort of priors about the environment would an ideal agent use?
Stuart Armstrong has written several blog posts about the specification of “reduced impact” AGIs:Say you construct a training set containing many outcomes filled with happy humans (labeled “good”) and other outcomes filled with sad humans (labeled “bad”). Within that subproblem, we currently focus on the utility indifference problem: how could you construct an agent which allows you to switch which utility function it maximizes, without giving it incentives to affect whether the switch occurs? It may seem easy: just iterate over each action, figure out which outcome the agent would get if it took that action, and then pick the action that leads to the best outcome. An attempt to answer the basic question, “When can ambitious projects to achieve unusual goals hope to succeed?”Program verification techniques allow programmers to become confident that a specific program will actually act according to some specification. We’re looking for talented, driven, and ambitious technical researchers for a summer research internship with the Center for Human-Compatible AI (CHCAI) and the Machine Intelligence Research Institute (MIRI).
Search Machine learning intern jobs.
Type theory also bridges much of the theoretical gap between computer programs and mathematical proofs, and is therefore often relevant to certain types of AI research.MIRI’s math research is working towards solutions that will eventually be relevant to computer programs. Inside the black box is a Rube Goldberg machine which takes the ball from the input chute to one of the output chutes.Most of modern mathematics is formalized in set theory, and the textbooks and papers listed here are no exception.
This research guide can help you get up to speed on the open problems being discussed on the IAFF.