Here’s my Saturday weekly reads. Links to what I enjoyed reading, with commentary.
1. AI GPUs crush Moore’s law. Human intuition appears to be hardwired to linear growth. Steve is a teenager, and if he grows an inch/year he’ll soon wind up as tall as his dad. But what if Steve grew exponentially? He’d match his dad at 6 feet tall in one year, be 12 feet tall next year, a mile high in 10 years, 1000 miles high in 20 years.
The point is an intuition resisting post by Dario Amodei and Danny Hernandez on exponential improvement in compute capability for training AIs. The doubling time is 3.5 months! Again, 3.5 months!! Compare that to 18 months for Moore’s Law (lame). Quote: “Since 2012, this metric has grown by more than 300,000x (an 18-month doubling period would yield only a 12x increase).” This is driven by GPU improvements and the increasing scale of AI investment. Also “We see multiple reasons to believe that the trend in the graph could continue.” Our minds are not built to grasp the implications of exponential growth. If nothing else, I do think it helps to internalize that the rate of improvement is far larger than Moore’s law. One consequence is we’ll see large progress in AI even if we assume (unlikely) no improved AI algorithms. If you click through the link, make sure to flip back and forth between linear scale and log scale on the graph, which brings the point home. link
2. Google Duplex skepticism. Here’s a story to pair with the above. The recorded demo of Google Duplex shown last week booking a hair appointment and restaurant reservation was very cool. But via John Gruber’s site it’s clear that demo was partly staged/edited. Leaving me more skeptical on whether this is demoware or a product ready for the real world. Caveats are: i) person answering the phone doesn’t give the name of the place, ii) there’s no ambient noise, iii) they didn’t ask for phone number/name of the person calling to book the appointment. Axios story. Gruber skepticism. Gruber dinging Mashable for stealing the story and getting it wrong. Also, finding the restaurant used in the demo via a twitter ask (which is interesting by itself).
3. Europa plumes. Besides Earth, the only known liquid water in our solar system is on Jupiter’s moon Europa and Saturn’s moon Enceladus. These moons are ice covered, with tidal forces keeping the watery oceans under the ice crust at liquid temperature. Enceladus is known to have plumes of water ice spurting out into space, when tidal forces occasionally crack the surface. But Europa is much bigger than Enceladus, with a miles deep ice crust that might not crack. So perhaps no plumes. But a new study based on 20 year old (newly analyzed) Galileo space probe data found a plume after all. In particular by matching a magnetic field change and increase in electron density with Hubble imagery. Excellent! NASA is planning to launch the Europa Clipper mission in 2022. And I was sort of bummed they didn’t choose Enceladus, since plumes seem to be the best hope for finding microbial life in our solar system. Now it’s all good. link
4. Ice core lead from Rome. New paper analyzes ice cores from Greenland, tracking lead pollution from Rome. The new study tracks 25,000 measurements, compared to 18 from a previous one from the 1990s. So it’s way better. The best part is the chart below, mapping a timeline of Roman history onto lead pollution. link
5. Homo naledi had a small but complex brain. New paper. Quote: “Despite its small brain size, H. naledi shared some aspects of human brain organization, suggesting that innovations in brain structure were ancestral within the genus Homo.” Human brains run around 1450 cc, Chimp’s 350, Gorilla’s 530, H. naledi was 510. So very small for the Homo genus.
What this brings to mind is Kevin Laland’s book Darwin’s Unfinished Symphony, which argues human evolution got kickstarted by getting into a virtuous cycle of cultural transmission. So brains evolved not to think, but to copy. Humans know not why we succeed. We merely copy those we believe are successful. Who in turn also don’t know why they do what they do. In this model, we first evolved a brain that was organized to culturally learn, and only later did that put selective pressure on making brains larger. So the sequence is 1. evolutionary selection for cultural transmission -> 2. modified brain (still small) to get better at cultural transmission -> 3. selection for bigger brains for better cultural transmission -> 4. language evolves to transmit culture better -> 5. culturally driven technology improvement.
Along those lines, I ran across this recent paper. From the abstract: “we focus on the growing body of work that treats language structure as emerging from the process of cultural transmission. We argue that a full recognition of the importance of cultural transmission fundamentally changes the kind of questions we should be asking regarding the biological basis of language structure.”
Minor aside. The careful reader will note how many quirks of human behavior are explained once you realize cognition evolved not to be rational, but to culturally imitate successful members of our preferred tribe. Thus my blog, and indeed much of contemporary politics.
Praxtime by Nathan Taylor 