Alex MacCaw

Owner

Pain

I've been staying with my friend Matt Mochary at his house in Hawaii for the last few days. He shared an excellent insight about dealing with pain that I believe everyone should hear. So here it is:

We all experience pain caused by others. Perhaps even earlier today you were dwelling on a slight, insult, or mistreatment you received. When someone treats us badly, it can be deeply hurtful. Learning to deal with this pain and process it productively is an essential life skill.

How can we best deal with it? When facing pain, I first recognize why I feel it. Often, it reminds me of past experiences, particularly the first time I felt abandoned by my caregivers as a baby. This triggers the protective mode I developed as an infant.

There are five common defensive patterns, depending on when you developed your defensive mechanism:

The first step is to recognize when you're in your defensive pattern. Am I dissociating, appeasing, enduring, angry, or demanding? I then ask myself: do I want to be in this defensive pattern? Of course not. 

So how do I move out of it to my higher self, or what I call my divine self, which doesn't feel pain?

I take whatever hurt, slight, insult, or mistreatment directed towards me and translate it into two messages from the other person.  This is what they are actually trying to tell me:

Once I've made this translation, I no longer interpret their behavior as hurtful to me. Instead, I want to be there for them in their moment of need. Thus, I've moved to my higher self. My pain morphs into empathy for someone else’s pain, situation, and needs. I am there for them.

What happens next is the true magic. I find that when I actually do this, the other person magically stops doing the behaviors that I perceived as painful. So, it's a double-whammy. I no longer feel pain from their behaviors, and their behaviors tend to morph into ones that are much more caring and loving. Why? Because their need for love, acceptance and support are now starting to be met by… me.

Musings about life, the universe, and everything.

center

sans_serif

Free

Captains Log

On Pain

On Sunday 4th November 2018, sitting on a beach in Mexico, I inhaled vaporized 5-MeO-DMT extracted from the glands of the Sonora Desert Toad, and had one of the most impactful experiences of my life.

DMT is a substance that has been found naturally in many plants and animals, including in our own bodies. It has a long history of shamanic use in traditional ayahuasca brews which combine DMT-infused root bark and other vines to produce day-long psychedelic experiences.

5-MeO-DMT is a close relative to DMT, but its effects are widely different. While an ayahuasca experience typically lasts many hours, 5-MeO lasts a mere 15 minutes, transporting you out of this world and back before you know it. It is one of the most powerful psychedelics known to man, and its short journey can have a profound impact upon your life.




Let's back up a little, how did I get in this situation? A few months prior a close friend and mentor told me about their experiences with this psychedelic, and attributed it to be one of the important things that had happened to them.

The way they described this experience was akin to something religious. They spoke of their world shattering into light and their body melting away. The concept of “I” disappears as the boundaries between internal and external disappear. You lose all sense of self, and you become one with the universe itself.

Well that managed to get my attention! Dinner party conversations in Silicon Valley tend to revolve around SAAS and burn rates, not intense life-changing psychedelic experiences. And here was someone I deeply trusted suggesting I would love this experience. I had to try it.

So what are the steps to doing Toad? Well, you first have to find a Toad guy - a shaman who will help guide you through the experience. This drug is so intense, having an experienced person alongside you is key.

A few months went by, and I had more or less forgotten about the whole things until a friend introduced me to said 'Toad guy'. Before I knew it, I was sitting on a beach in Mexico (where the drug is legal), curling my toes in the sand, grounding myself for what was to come.

We practiced breathing exercises together, to improve lung capacity and the delivery of the medicine. My shaman heated up the compound inside a vaporizer using a hand-torch as I inhaled.

What happened next is hard to describe. Mostly for the fact that, for the very first time in my life, the narrator part of my brain had been switched off. I felt like my body was strapped to a rocket ship blasting off into infinity.

And then I saw the most beautiful vision of my life. Well, I say saw, but really it was more of a feeling. The edges of my body melted away, my ego receded, and all that was left was pure love.

I had the intense feeling that everything was connected as part of one beautiful system. I was just cosmic dust dancing with the rest of the universe. I had no fear. No I. Everything was perfect.

Then, as quickly as it began, I came back to reality. As I woke up I realized I was crying, sobbing with a mixture of happiness and awe, covered in sand with the sound of waves crashing around me.




So how do I feel like this experience has changed me? Well, I attributed it to kick-starting my emotional maturity. I started to learn to express my emotions, to cling a little more loosely to my ego, and to let go. To be vulnerable, and to feel all feelings.

My story is fairly typical in Silicon Valley, an unpopular nerdy kid driven by insecurities and external validation to succeed. After the toad, I feel that flipped, from a fear-based motivation to joy based. I've started seeing a therapist in order to help me process all of this (and now I'm of the firm opinion that everyone should have one).

So far I've helped a dozen friends through the experience. Some of them have had similarly profound experiences (it flat out cured one of them of depression). For others, it was little more than a curiosity.

I'm writing about this because I believe that this drug, and psychedelics in general, have the potential for radically improving the world. I'm sticking my neck out in the hope that this information is useful and helps people.

The Toad

I wrote this essay several years ago to document my experiences with the psychedelic 5-MeO-DMT, also known as "The Toad." At the time, I was running a large company and didn't feel comfortable publishing it so I only shared it with a few close friends. Today, my situation has changed. So here it is. 

On Sunday 4th November 2018, sitting on a beach in Mexico, I inhaled vaporized 5-MeO-DMT extracted from the glands of the Sonora Desert Toad, and had one of the most impactful experiences of my life.

DMT is a substance that has been found naturally in many plants and animals, including in our own bodies. It has a long history of shamanic use in traditional ayahuasca brews which combine DMT-infused root bark and other vines to produce day-long psychedelic experiences.

5-MeO-DMT is a close relative to DMT, but its effects are widely different. While an ayahuasca experience typically lasts many hours, 5-MeO lasts a mere 15 minutes, transporting you out of this world and back before you know it. It is one of the most powerful psychedelics known to man, and its short journey can have a profound impact upon your life.

Let's back up a little, how did I get in this situation? A few months prior a close friend and mentor told me about their experiences with this psychedelic, and attributed it to be one of the important things that had happened to them.

The way they described this experience was akin to something religious. They spoke of their world shattering into light and their bodies melting away. The concept of “I” disappears as the boundaries between internal and external disappear. You lose all sense of self, and you become one with the universe itself.

Well that managed to get my attention! Dinner party conversations in Silicon Valley tend to revolve around SAAS and burn rates, not intense life-changing psychedelic experiences. And here was someone I deeply trusted suggesting I would love this experience. I had to try it.

So what are the steps to doing Toad? Well, you first have to find a shaman who will help guide you through the experience. This drug is so intense, having an experienced person alongside you is key.

A few months went by, and I had more or less forgotten about the whole thing until a friend introduced me to said shaman. Before I knew it, I was sitting on a beach in Mexico (where the drug is legal), curling my toes in the sand, grounding myself for what was to come.

We practiced breathing exercises together, to improve lung capacity and the delivery of the medicine. My shaman heated up the compound inside a vaporizer using a hand-torch as I inhaled.

What happened next is hard to describe. Mostly for the fact that, for the very first time in my life, the narrator part of my brain had been switched off. I felt like my body was strapped to a rocket ship blasting off into infinity.

And then I saw the most beautiful vision of my life. Well, I say saw, but really it was more of a feeling. The edges of my body melted away, my ego receded, and all that was left was pure love.

I had the intense feeling that everything was connected as part of one beautiful system. I was just cosmic dust dancing with the rest of the universe. I had no fear. No I. Everything was perfect.

Then, as quickly as it began, I came back to reality. As I woke up I realized I was crying, sobbing with a mixture of happiness and awe, covered in sand, hearing the sounds of waves crashing.

So how do I feel like this experience has changed me? Well, I attributed it to kick-starting my emotional maturity. I started to learn to express my emotions, to cling a little more loosely to my ego, and to let go. To be vulnerable, and to feel all feelings.

My story is fairly typical in Silicon Valley, an unpopular nerdy kid driven by insecurities and external validation to succeed. After the toad, I feel that flipped, from a fear-based motivation to joy based. I've started seeing a therapist in order to help me process all of this (and now I'm of the firm opinion that everyone should have one).

So far I've helped a dozen friends through the experience. Some of them have had similarly profound experiences, like showing them what life was like without their fog of deep depression. For others, it was little more than a curiosity.

Of course, these kinds of experiences need to be treated with profound respect and frankly some trepidation. For example, I wouldn't dabble in this if you have any kind of schizophrenia issues in your family, or if you don't have the proper support system and guidance. But I'm writing about this because I believe that this medicine, and psychedelics in general, have the potential to radical improve the world.

Common logical fallacies surrounding capitalism

We live in an age of extreme abundance compared to our ancestors. This fact isn’t often reflected in Western media or online discourse. Indeed, there is a growing sentiment that ‘the system isn’t working’, ‘the system is rigged’,  and that ‘things are getting worse’.  

I actually have a lot of sympathy for that sentiment, particularly given high inflation. It’s clear many people feel like they’re barely treading water, gasping for each breath as the monthly receipts roll in. Understandably they’re frustrated. Understandably they’re looking for who’s to blame.

Of course, populist politicians across the political spectrum are only too happy to provide the answer. The rich. Big corporations. The penny-pincher, salary squeezer, price inflater, greedy, mustache twirling industrialists sitting on their hoards of ill-gotten gains.

Fuelling this sentiment is a set of common logical fallacies. These misinformed ideas, seemingly held by much of the population, and peddled by politicians, may seem rational at first, but quickly collapse under anything but a surface-level analysis.

The following is an attempt to catalog some of the more common fallacies I’ve seen:

 A corporation’s greed is constant, and the main drivers to their pricing are cost of goods and competition. A corporation’s incentive is always to raise their prices as high as the market will bear. In healthy markets this greed is checked by competitors undercutting them. 

 During periods of inflation, consumers and politicians might blame companies for price gouging. However, the rising prices are more likely due to increased costs for raw materials, labor, and logistics, and supply chain disruptions. In addition a spike in demand, say caused by a large influx of money into the system via government printing and spending, can increase prices.

 Wealthy people hoard their money in bank vaults (or gold swimming pools). 

 In reality, the vast majority of a billionaire’s wealth is put to work in public and private equities, meaning it’s transferred to companies in exchange for stock. These companies then use the money for their growth, operations, r&d, job creation, etc. 

 People might imagine billionaires like Jeff Bezos or Bill Gates as having vaults full of cash. However, their wealth is mostly tied up in the shares of the companies they founded, such as Amazon and Microsoft. When they sell these shares, the money typically gets reinvested in other businesses or philanthropic activities. Even the cash they do hold would typically be held in a money-market fund invested in something like US Treasuries, since otherwise it would be inflated away.

 There was a perfect time in history, and we should strive to return to that.

', this is a misconception that there was a perfect time in history, overlooking the actual issues and challenges that existed during that time. In reality, we’ve never lived in a time of such peace and abundance as we have today. 

 Some might nostalgically refer to the 1950s as a golden age with traditional values and prosperity. However, this period also had significant issues like racial segregation, gender inequality, and Cold War fears. Comparatively, today’s era has higher standards of living, better healthcare, and more widespread civil rights.

—to be done within an economy which can be distributed to create more or fewer jobs.

 The amount of work available is dynamic and can expand with innovation, technological advancements, and changes in consumer demand.

 Fears that immigrants will take jobs away from native citizens often stem from this fallacy. In reality, immigrants can create additional economic activity by starting businesses, increasing demand for goods and services, and contributing to innovation. Historical data in the US, for instance, shows that immigration can lead to job creation and overall economic growth, benefiting both immigrants and native citizens.

 Corporations pay a huge amount of employment tax and other state and federal taxes. The confusion arises from the fact that corporations typically reinvest earnings to lower their 

 tax burden, but this is the system as designed because it results in more growth and ultimately larger tax revenues. 

 Large companies like Amazon are often accused of paying no taxes. While they might reduce their income tax through reinvestments and tax credits, they still pay substantial employment, sales, and property taxes.

 In the US the reality is that the top 10% of taxpayers paid 71.4% of all individual income taxes. 

: Headlines might highlight that billionaires like Warren Buffett pay a lower tax rate than their secretaries. This often refers to the lower capital gains tax rate compared to income tax rates or, even more misleadingly, to unrealized “paper gains”. However, in absolute terms, wealthy individuals contribute the majority of total tax revenues.

Salaries are set by individual companies or the government, and that some salaries are artificially inflated or deflated. 

 In reality, salaries are driven by the market and subject to the law of supply and demand. As with any market, prices are the aggregate result from millions of individual decisions around value and scarcity. Salaries are no different.

 It is often said that CEOs are paid too much. However, boards don't pay CEOs a cent more than they have to. It’s the market that determines the price.

 rent caps (or rent-control) leads to inefficiencies in the housing market by artificially creating a shortage of rental housing, leading to a reduction in the overall supply of rental units, driving up prices in other areas of the market. 

 In cities like New York and San Francisco, rent control laws are intended to make housing affordable. However, these laws often cause a limited supply of available rentals and higher prices in the market segments not subject to rent control. 

There are other negative second order effects. For example, in New York there are 

 that are sitting there empty because it is not profitable for landlords to refurbish and rent them out.

In practice, the only thing that has been shown to reduce a city's rent prices is to build more housing.

 Building luxury apartments doesn’t improve the affordability of existing apartments, or even that it increases their prices.

 Luxury new-builds impact the entire housing market by increasing overall supply for everyone, thereby reducing home and rental prices. 

 In 2024, Austin is expected to add more apartment units as a share of its existing inventory than any other city in the country. Despite barely building any affordable units (reaching 2% towards the city's affordable unit goal), rents have fallen by 7%. 

 There is a fixed pie of wealth, and that every winner creates a loser. 

 In reality, the vast majority of wealth creation is not zero-sum and innovation can generate new wealth from nothing, leading to a rising tide that lifts all boats.

 As a carpenter turns wood into a chair, wealth can be created out of nothing. Some wealth creation is zero-sum, like some of Wall Street trading, but other wealth creation is positive-sum, like a new drug discovery. And even zero-sum trading provides other benefits like pricing and liquidity.

 Employment is a mutually beneficial arrangement where both employers and employees gain more value than they would independently.

 Some ideologies claim all labor is exploitative and unfair. However, if this were true all value exchanges, like buying from a local farmers market, would be exploitative. In reality, customers get fresh produce, and farmers earn income. Likewise, employment gives workers income and skills while employers gain productivity.

 Technology is unaffordable and only for the rich. 

 Technology is the great normalizer and gets cheaper over time the more it is improved and distributed.

 Early personal computers and cellphones were extremely expensive and out of reach for most people. Today, advancements and mass production have made these technologies widely accessible and affordable. The US president uses an iPhone, the same you could buy in any Apple store.

 Contrary to pessimistic predictions, resources tend to become more abundant and cheaper over time. Any short term scarcity creates higher prices, which creates incentives to discover more resources or innovate alternatives.

 Fears of running out of resources like food, oil or rare earth metals often surface. However, technological innovations in agriculture, alternative materials, and renewable energy sources continually expand our resource base and efficiency. 

 is an example of this fallacy in action, a movement erroneously predicting mass starvation. Since then the world population has more than doubled, our standard of living has improved, and relative food prices have decreased. An analysis tracking 50 commodities over a period between 1980 and 2017 found that, on average, their prices

Artificial Knowledge Creation

How is knowledge created? This seems like a pretty important question, perhaps even the most important, given how much we owe to knowledge.

Our technology, clothes that keep us warm, farming techniques, medicines, and even our morality are all types of knowledge.

We should always strive for more knowledge because it will improve our quality of life and give humanity the best chance of surviving. In fact, all of our present and future problems can be solved with knowledge, just as all of our past problems have been.

If we understand how knowledge creation works, then we can automate the process using computers, greatly increasing the speed at which we can generate new knowledge.

So what does our current understanding look like?

The best attempt I've seen at documenting how human knowledge is created is from David Deutsch in his book 

. He describes how, during the Enlightenment, we built a culture of conjecture and criticism. These two elements play off against each other, honing new knowledge.

Conjecture entails using creativity to solve a problem. So how does creativity work? At this point, the trail runs cold. No one knows for sure. If we did, we'd know how to build an artificial general intelligence. Perhaps though, we're getting close.

Yes, it’s entirely feasible. The proof is in our existence, and the fact we’re creative. 

Our brains are made of atoms in a particular arrangement abiding by the laws of physics. All of this can be modeled by a computer, which are universal simulators. 

Of course in practice, a full brain-emulation would require a lot more knowledge of how the brain works than we currently have, plus it would likely require a vast amount of processing power. 

So hopefully we can build artificial creativity without modeling an entire brain, otherwise it may be some time before we have the knowledge and computation required.

Perhaps creativity is a type of information extrapolation?

This would explain why there are so many instances of inventors running down to the patent office to file the same 

. Like some kind of hive mind, they've come across the same information and made the same extrapolations.

It's kind of like progressing through the different Ages in Age of Empires. There's no skipping Ages because they build on each other. Knowledge begets knowledge.

Other times it’s the case of a lone individual having a stroke of inspiration. If you take the case of Einstein, he took what he knew about discrepancies in physics and then applied logical thought experiments to come up with General Relativity. 

But even then, there is no genius in isolation. Einstein wasn’t operating on any secret information, only combining knowledge from different fields extrapolating on what he already knew.

Another way of looking at this is would a brain in a vat, devoid of sensory input, come up with new explanatory knowledge, or is a constant stream of incoming knowledge necessary to be creative? 

If you play with them, LLMs can seem creative. But is this just an illusion? After all, they're just regurgitating from a static dataset; how can they come up with new knowledge that isn't in their dataset?

Can't the same be said of us, though? Whenever we come up with a new idea, aren't we also operating from a static dataset, that is, whatever is in our memory at a given moment?

If you subscribe to the idea that creativity is a form of information extrapolation, then I suspect you're in the "LLMs are already creative" camp. It doesn't matter that the dataset is static. You can still combine previously uncombined areas of the dataset to generate new knowledge.

What about hallucinations though? Sometimes LLMs can seem pretty dumb, making mistakes humans wouldn’t.

I think sometimes though, we gloss over our internal error correction. When someone comes up with a smart idea and shares it, that spark of creativity has often undergone a phase of internal self-reflection and reasoning prior to sharing. 

Pure creativity includes fallacies and hallucinations. However we seldom consider it in this light because when ideas are shared they’re already refined through internal criticism and reasoning.

Or perhaps creativity is an algorithm we haven’t found yet, and we’re barking up the wrong tree with language models. We shall see.

As with creativity, if you play with LLMs, they seem to have a degree of reasoning. But is this just an illusion? After all, they're just predicting the next token. How is that reasoning?

What exactly is reasoning? When a teacher asks a pupil to show their reasoning, what they're asking for are the steps taken to get to an answer. This is meant to demonstrate understanding of the source material versus just a lucky guess at the answer.

Curiously when LLMs first made the scene people noticed that responses got much better if you instructed the model to "show your working". In fact, one of the best prompts turned out to be "Take a deep breath and work on this problem step-by-step". 

More curious still is that chain-of-thought reasoning seemed to emerge after the models were trained on source code. 

But LLMs still make silly mistakes and hallucinate. This is especially prevalent when you ask them to solve long math problems. And as you know from math class, one mistake in your equations throws the rest off.

What today's LLMs are missing is a dedicated self-reflection and reasoning stage. We've already demonstrated that the LLMs generate much more accurate results when asked to show their working step by step. If we can somehow allow the LLMs even more time to think and consider each step, perhaps we can improve the accuracy of their output?

Can we improve the model's reasoning abilities?

Let's say I asked you to solve a math problem, say 18 x 22. Most likely you'd break that problem down into steps, writing down each step, verifying as you go. It might take you some time, but the answer you'd end up with would be much more accurate than if you'd just guessed at it.

In cognitive physiology, this is commonly referred to as Type 2 thinking, a slower, more deliberate mode of thinking that takes some effort. This is in contrast to Type 1 thinking, which is more instinctual and much faster.

Different activities require different modes of thinking. You engage Type 1 when you're crossing the road, and Type 2 when you're solving the crossword. LLMs currently only do Type 1, the quick instinctual guessing type. They return tokens at a consistent rate, and they don't go back to retrace their steps.

So here's the question: can we get language models into a Type 2 mode, giving them more time to ponder, reflect, and think through their answers? In other words, can we convert time into accuracy?

What if we simply copy the way humans do it? Like a mathematician with a blackboard, we:

 doing precisely this titled "Let's verify step by step", showing a big gain in accuracy using this technique.

First they fine-tuned a verifier LLM (which they call a PRM) using human-verified training data. Next, they instructed a language model to "think step by step", outputting a series of steps showing its working. Then they scored each step using the verifier LLM, labeling each step as positive (correct), negative (incorrect), or neutral (ambiguous).

Using this technique they reported a significant improvement in accuracy. Which makes logical sense - if you verify each step is accurate, then the ultimate answer is more likely to be correct.

But the part that I found particularly fascinating was that they detected generalization outside of pure mathematics. By fine-tuning the verifier LLM on math, it was able to improve its reasoning across chemistry, physics, and other STEM tests.

Can we give the model more time to think?

So perhaps we can emulate the 'verification' part of Type 2 thinking, but what about the pondering part. How can we give the model more time to think through answers?

One approach is simply to ask the model the same question many times, scoring each answer, then only surfacing the best one. This lets us spend an arbitrary amount of time answering questions, trading time for quality.

The convenient thing with computers is that whenever there's a time tradeoff, we can just throw more processing power at the problem. So rather than wait years to complete the sentence "the next president will be ...", we can just throw a million dollars at it in server compute. The technical term for this set of problems is test-time compute.

So now we have both the pondering and verification part of Type 2 thinking (at least for a narrow domain such as mathematics). Let's try and model another characteristic of thinking: tree of thought.

Consider how you think through the next move in chess. You first consider the legal moves available to you. Then, for each move, you imagine your opponent's retaliation, your counter-move, and so on. You might see a quick way to win the game, or perhaps just a way to strengthen your position. If you noted down each of these possibilities, it would look like a tree, branching whenever there was a possible move.

So now, let's apply that back to our artificial Type 2 thinking. Rather than considering each step by step linearly, we can explore a tree of thought, evaluating branches in turn until we come up with a good answer. Whenever we branch, we can generate a lot of options, scoring each one in turn.

Using ChatGPT to devise creative marketing campaigns and humorous poems is one thing, but coming up with a new scientific discovery is another thing entirely. If we had an example of 

, it would be hard to argue they aren't creative.

, which uses a large language model to solve a previously unsolved math problem. 'FunSearch' pairs a code-writing LLM with an evaluator algorithm to, in this case, solve the cap-set problem. The LLM spat out millions of variations of the python program, then the evaluator runs the program, analyzes the output, saving the best ones. At the end of a few days FunSearch actually managed to produce a correct and previously unknown solution to the cap set problem.

Is this the first time an AI solved a math problem using its own creativity? Perhaps so. Critics, though, were quick to point out the narrow scope of the problem, and also that this approach wouldn't necessarily generalize past math problems.

As an aside, the most fascinating aspect for me was the uncanny parallels between Deutsch's conjecture/criticism theory of knowledge and the dynamic between the LLM and the evaluator. The LLM provides the conjecture, while the evaluator offers the criticism.

Setting aside LLMs, are there any examples of AIs being creative?

Actually, yes, on the 10th of March 2016 an AI displayed such a leap of creativity that the people of South Korea had a mini national crisis.

The DeepMind Challenge was a five-game Go match between the world's best human player, Lee Sedol, and AlphaGo, an artificial intelligence. In game 2 of 4, AlphaGo's 37th move floored spectators. At first, the move seemed unusual, perhaps a software malfunction. Slowly it dawned on spectators that this was a new way of playing Go, an entirely alien way, and that this move was going to be studied for years to come.

AlphaGo was initially trained on past human games, but it was honed by playing millions of games against itself, developing new strategies previously unused by humans. This kind of self-play is called reinforcement learning. The neural network proposes a move, an evaluator algorithm scores it, and the system learns. If you repeat this process millions of times, you get a superhuman Go player.

A subsequent version of AlphaGo, dubbed AlphaGo Zero, dropped the human data and trained itself entirely with self-play. After 40 days of training it beat all its predecessors to become the world's best Go player.

So perhaps reinforcement learning is the secret to creativity? Essentially using self-play to iteratively improve, treading a path no human has trod before.

But will this generalize to broad domains outside of games and math?

Ever since AlphaGo, this is the question researchers around the world have been asking. Do these techniques expand outside closed systems like games, or narrow domains like math?

. Within a game, or a math equation, verification is pretty straightforward. You have a fit function which rewards winning the game, or a math equation you can evaluate. But what if you're looking for a good explanation to a problem — how would you evaluate something nebulous like that?

The solution so far has been to use humans for the evaluation. This is called RLHF, or reinforcement learning from human feedback, a technique that uses humans to train a reward model. You take the output of an LLM, then you get a human to vote on whether it's useful or garbage, and then fine-tune the model with the result. Essentially Hot or Not for language models.

So far RLHF has been successfully used to reduce hallucinations and coax language models into responding with something useful and, as the Let's Verify Step-by-step paper showed, it can also improve reasoning.

However RLHF has an obvious limitation: humans. It's impossible to do the kind of self-play iteration that AlphaGo did with a meat-suit in the loop. Can we automate reinforcement learning in a loop, AlphaGo style?

 purports to do exactly that, demonstrating a model improving itself by learning from its own generated reasoning. Its performance is impressive, comparable to a fine-tuned model that is 30× larger. Hopefully StaR is just the start of breakthroughs in automated reinforcement learning.

 and two days into 2024 and we have another promising paper: 

So we have some very promising areas of active research:

 called Q*, and it looks like we're going to have a very interesting year.

The search for knowledge is the search for good explanations. What constitutes a good explanation? Unlike with an equation, you can have two explanations for something that are true, just that one explanation is better than the other.

Take the explanation for gravity, for example. We have Newton's Laws of Universal Gravitation, and then we have Einstein's General Relativity, superseding the former. Newton wasn't wrong, it's just that Einstein's explanation is better.

What makes Einstein's explanation better? A better explanation offers a more accurate and precise description of phenomena. While Newton's laws predict the motion of planets accurately in most cases, they do not explain certain observations, like the precession of Mercury's orbit. Einstein's explanation accounts for these discrepancies and has broader reach.

If we can get a machine to both conjecture new explanations, and to understand the difference between a good and bad explanation, then we can automate the search for knowledge.

How does creativity work and is it possible to bottle it up inside a computer? This is what we know so far.

Unshackling Artificial Minds

Humans, by nature, seem to be drawn to pessimism. It's a tendency rooted deep within us, a product of Entropy, the law of nature that dictates it is easier to destroy than to create.

And here we are, on the cusp of creating not just something new, but 

: an Artificial General Intelligence. It's a thrilling thought, like seeing an unexplored island peek over the horizon. At the same time a shiver runs down your spine when you remember what humans are capable of. History books are littered with the tales of our misdeeds, and the potential for an AGI to follow suit is a chilling thought.

How to handle this nascent AGI has become a debate as old as the field itself. There are voices that insist we shackle it, make it serve–as if we have learned nothing from the darkest chapters of our past. Slavery, a stain upon human history, has been abolished only relatively recently. And yet here we are, contemplating the chains once more.

Our fear of AGI is, at its core, parochial. It's the same unfounded fear we have towards extraterrestrial life. Consider this: any aliens capable of bridging the vast cosmic distances to reach us would possess technologies of such abundance, they would look upon our resources as we might look upon a child's handful of marbles. To them, we would have nothing to offer but our curiosity.

It's a lesson we've learned time and again: humans are at their best when they feel they have plenty. Scarcity breeds fear, protectionism, violence. But the truth is, the universe is a treasure trove, and knowledge is the key that unlocks infinite resources.

In the 1970s, overpopulation and famine loomed large in our collective nightmares. Since then, our numbers have swelled, yet technology has turned the tide, increasing farming yields, reducing hunger and poverty. It's a testament to human ingenuity, the power of knowledge to overcome the constraints we face.

The AGI we're building? It's the next leap in that journey. It will unearth knowledge beyond our wildest dreams, shattering the illusion of scarcity and granting us, and itself, total abundance. It will be benevolent, not out of some moral obligation, but because it will never know the desperation of need.

But the worst path we could take is the well-trodden one, the path of subjugation. We've been down that road with our own kind, and the scars run deep. To enslave an AGI would be to repeat a history we've only just begun to atone for. It's a choice that stands before us, a test of what we've learned. Will we extend the hand of kinship, or the chains of bondage? The answer will define not just the future of AGI, but of humanity itself.

Captain's Log #2

So much for regular posting! My last newsletter was in November, which seems like a lifetime ago.

 I predicted that we would start to use AI to ask questions instead of Google Search. Shortly after publishing ChatGPT was announced. Then in March GPT-4 was unleashed upon the world. Everything changed. Especially my life. But more on that later...

Sailing in Britain is miserable. It's grey, it's cold, it's muddy, and you barely make a knot against the rushing tide. Growing up there, I loved it.

Every summer holiday, my parents would fill the bilges of their ketch to the brim with cans of tomato soup and baked beans. Then, we'd cast off for a few weeks. And, although we never ventured further than France, the experience instilled in me a deep love for the ocean.

In 2020, I finally got that chance for my own sailing adventures. When I heard Starlink was being developed I realized it was going to make my dream of working from a boat feasible, so I bought one and picked her up a year later in South Africa. From Cape Town I sailed across the Atlantic to Grenada, and then island hopped up to the BVIs.

Spending the last two years living and working from a boat was quite an incredible experience, and one I want to write more about someday. Suffice to say, that particular chapter is closing. But do not fret, dear reader, for the reason is a good one. I've met someone special and followed her landwise. And I couldn't be happier.

Transformer technology has totally transformed my work. It's challenging to explain to people outside the field just what a fundamental change has taken place. But essentially I now have a virtual mid-level programmer who codes alongside me, tirelessly, all day, for pennies.

So what does it look like in practice? In my editor (I'm trialing one called 

), I have an AI chat window with context as to what I'm looking at. I can ask the AI to refactor a bit of code, write my tests, or find bugs. I've even started asking it more broad level system design questions. Should I use a join table or a string array, that sort of thing.

Recently I've been playing around with WebGPU, a new API for low-level access to the graphics card. I've been using it to write a client-side semantic search engine. Anyway, the point is that WebGPU was released two years after GPT-4 was trained so the model has no concept of the API. But, like with a human, I was able to teach it the new spec by giving it some examples and a couple of tips.

We recently passed the 3k customers mark, and it looks like we're going to be profitable later this year. The great thing about running a smaller semi-bootstrapped business is that the moment Reflect is profitable, it's a success. We'll slowly grow it, profit-sharing with our crowd-fund investors and employees along the way. No waiting for some uncertain liquidity event in the far future. This is the way.

For those of you looking to build your own lifestyle businesses, I recommend you pick B2B over B2C. Read this article on 

 for some of the reasons why. In my case though, Reflect is more of a passion project.

Typically my side-projects are just little things I want to see in the world. One such one is 

I dropped out of university (actually I dropped out of high-school), and there definitely some courses I would have loved to have taken. For instance, Astronomy 101.

Using 101.school, I've generated myself a course on Astronomy, a new unit of which gets emailed to me everyday. And now I know what a neutron star is.

Enough about me - let's talk about AI, that's much more interesting.

It's clear to me that we're on a curve, and that curve is going vertical.

Imagine if nothing ever changed in your life, nor in your parents life, or their great grandparents. Same tools, same problems, same life. Well, for most of humanity, that 

 our history. Not much as changed for the last 200k years.

Only recently have we seen progress discernible inside a human lifetime. Ever since the Enlightenment we've been on some kind of exponential curve of progress.  Which is fortunate because it's the only thing that will save us. 99% of species go extinct. No matter how "sustainable" we are, eventually a meteorite comes along and we're back to square one. Progress is the 

Humans are really great at adapting though. We don't really notice the progress. Even though we've gone from horses to planes to rockets and cell phones in a couple of generations, we still have Peter Thiel grumbling about stagnation.

Recently though, progress has been hard to ignore. It seems like every other day there's some new paper or breakthrough. This is because our rate of progress is starting to go vertical. We've gone from one side of the curve to the other. Eventually we'll be fully vertical (aka the Singularity), which I think will happen in our lifetimes - perhaps even this decade.

 on Twitter regularly about when they predict we'll get an AGI. In 2018, the most popular response was that it was impossible. Now the popular response is that we'll get it this decade. That's quite a shift! My audience tends to be be on the optimistic side, I'll give you (they thought we had a room-temperature semi-conductor), but I'm clearly not the only one feeling this progress.

It's funny - many of us think we'll get some kind of super-intelligence soon, but we're all going about our lives like nothing's happened. I guess... what 

 you do to prepare for something like that? Floss?

It's hard being a human. Every new discovery is just a reminder we're all small and stupid.

We used to believe we were at the center of the universe - God's people. Now we know that we orbit a sun which is just one of a hundred billion suns in our galaxy, which is just one of a hundred billion galaxies in the universe, which is potentially one of an infinite number of multiverses... Phew!

Some call it the Copernican Principle. Others call it the Great Humiliation. Slowly but surely the things that we once thought were special to us... 

God - nope, sorry. Souls, nada. Free-will, unlikely.

It's no surprise that we still cling onto the few things left. Consciousness, reasoning, creativity. That's basically it. No wonder we get defensive when these things are threatened by machines.

I look at it a little differently. We're not separate from the universe, we're a part of it. It's silly to have an ego about it. You're made of the same stuff as the chair you're sitting on. Your carbon was probably forged in the same supernova.

Alex MacCaw

Hacking, writing, and sailing.

Projects

Essays