Computational Power and AI
Jürgen Stowasser
January 2024
Compute as a Noun
"We're currently training our next-gen model Llama 3, and we're building massive compute infrastructure to support our future roadmap, including 350k H100s by the end of this year -- and overall almost 600k H100s equivalents of compute if you include other GPUs." Marc Zuckerberg announcing the development of the open-source large language model Llama 3, 18.01.2024 https://www.instagram.com/reel/C2QARHJR1sZ/.
“Right now in AI, what worries me is we don’t have the resources to make sure that academic AI continues to be a centre of gravity. Because if we lose that centre of gravity, then the other centre of gravity is driven by capitalism, Public-sector investment in AI is so abysmal. Not a single university today can train a ChatGPT model . Academia cannot fully develop its own versions so that it can be used for more open scientific research. That is a problem.”
Fei-Fei Li, Human-Centered AI Institute at Stanford (‘Maths is pretty clean. Humans are messy’, Financial Times Dec 15, 2023)
The term "artificial intelligence" means a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations or decisions influencing real or virtual environments. Artificial intelligence systems use machine and human-based inputs to— (A) perceive real and virtual environments; (B) abstract such perceptions into models through analysis in an automated manner; and (C) use model inference to formulate options for information or action.. National Artificial Intelligence Initiative Act of 2020
AI is the ability of a machine to display human-like capabilities such as reasoning, learning, planning and creativity. AI enables technical systems to perceive their environment, deal with what they perceive, solve problems and act to achieve a specific goal. The computer receives data - already prepared or gathered through its own sensors such as a camera - processes it and responds. European Parlament, 2020/2023
"AI socio-technical systems": Technology is always part of society, just like society is always part of technology. This also means that one cannot understand one without the other. Technology is not only design and material appearance but also sociotechnical; that is, a complex process constituted by diverse social, political, economic, cultural and technological factors. EU-U.S. Terminology and Taxonomy for Artificial Intelligence, 31 May 2023
It is important to understand the meaning attached to this adjective 'heuristic' which increasingly permeates the Artificial Intelligence literature: it means that the program stores and utilises a large amount of knowledge derived from human experience in solving the type of problem concerned. Thus it depends critically on data derived through the use of human intelligence so that the widespread view that only heuristic methods are effective is a serious setback to more extreme versions of the Al philosophy. Lighthill Report "Artificial Intelligence - A General Survey", 1973
By the time we are six years old, we recognize more than 10⁴, and keep learning more throughout our life. As we learn, we organize both objects and categories into useful and informative taxonomies and relate them to language. Replicating these abilities in the machines that surround us would profoundly affect the practical aspects of our lives, mostly for the better. Certainly, this is the most exciting and difficult puzzle that faces computational vision scientists and engineers in this decade.
Li Fei-Fei, Rob Fergus, and Pietro Perona: One-Shot Learning of Object Categories (2006)
GPT-4 draws a unicorn with LaTeX code:
Sparks of AGI(?)
Nightshade (SAND Lab, Univ of Chicago
↓