Quick Takes

Meanings of political identities shift dramatically based on context, and you can't manually confirm the beliefs of everyone present at your 'gathering of people with x political identity'. To the extent that your political identity is based on Real Beliefs with Real Consequences, you should expect not to have much in common with many other people who declare the same identity when you move to a new place (or corner of the internet). Example: In rural Southeast Texas, Confederate flags are a common sight, and my geometry teacher once told us about a cross burning he witnessed (which a few students murmured we really ought to bring back). The majority of people genuinely hold at least one belief that, to many of my coastal-elite-descended friends, would seem comical. E.g., women should never have jobs and should rarely speak (especially in public), men with long hair are wanton or gay or trans or both, beating children (not like 'spanking' but like 'anything short of broken bones') is not only fine but your duty as a father, weed overdose not only can but will definitely kill you, megadoses of zinc can cure cancer, the covid vaccine is the mark of the beast from the book of revelation, high school football ought to be the most important thing in your life and, if it isn't, you are not just odd but untrustworthy, and abortion doctors force-feed fetuses to geese to make fois gras for gay New Yorkers (of which the force feeding is the only ethical component). Okay, I made up the last one, but the rest are actual positions I've heard espoused hundreds or thousands of times by people I met between the ages of 14 and 18. Also many people talk like this, and everyone's a 'libertarian'. My mom's from a conservative California family with environmentalist sympathies, and we had something like 60 percent overlap in our views prior to the Texas move. However, I soon found that everyone around who wasn't liable to drop one of these devastating truth bombs on me thought of t

New reacts available only to paid users of LessWrong Premium (not you freeloaders) facilitate frictionless, borderline-telepathic communication. ‘I will NEVER change my mind’: Use this react to assert that you’re content with exactly how wrong you are (which is not at all), and that the case is permanently closed on this matter, so far as you’re concerned.1 ‘EY Stamp of Approval’: Use this react to assert that, on your personal authority, Eliezer Yudkowsky agrees with the contents of the comment, rendering it beyond reproach. ‘NOT EY Approved’: Use this react to assert that, on your personal authority, Eliezer Yudkowsky disagrees with the contents of the comment, rendering it immensely reproachful. Users who accrue too many ‘NOT EY Approved’ reacts will have their accounts suspended (although actual thresholds here have yet to be set).2 ‘May as well be AI’: Use this react when you’re indifferent to whether or not a statement was generated by AI because shit, it may as well be. You’re ignoring it either way. ‘Have you even read plane crash?’: Use this react when your interlocutor’s unfamiliarity with prior literature is clearly on display. ‘China Hawk’: Use this react to assert American supremacy and insist that its recipient is derelict in their duty to ensure the preeminence of the greatest country in the history of conscious life from here to the other side of the singularity. ‘Toilet’3 ‘Sure, buddy’: We all know that the optimal amount of mental masturbation is non-zero. But some reach far beyond the zone of optimality and into the depths of their pants to produce truly monstrous works of self-gratification. Previously, one was compelled to express such an opinion obliquely on LessWrong, or else shatter the decorum of the space and open oneself up to similar critiques. In Beta, we now have the power to quietly acknowledge the reality of the situation, without derailing the gratification itself. ——————————————————————————————— 1This has replaced the ‘I b

There was some discussion recently about the uptick in object-level politics posts and whether this is desirable or not. There's no rule against discussing politics on LW, but there is a weak norm against it, and topical discussions have historically tended to be somewhat meta and circumspect. I think the current situation is basically fine, and it's normal for amount of politics discussion to ebb and flow naturally as people are interested and issues become particularly salient. That said, here are a couple of potentially overlooked reasons in favor of more object-level politics discussion: 1. To build skill. Discussing politics productively is a skill that requires practice and atrophies without use. "Politics is the mindkiller" never meant that you should not discuss politics at all; it means that discussing politics is playing on hard mode. But sometimes playing on hard mode is the best way to level up. I suspect the skills needed to discuss politics productively overlap with lots of other important rationality skills. 2. To create common knowledge and avoid conflationary alliances. It can be confusing or disconcerting to not know what kind of common background assumptions the commentariat takes for granted when discussing something politics-adjacent. This is a problem somewhat unique to LW and not necessarily a bad thing; most other places on the internet skew too far in the opposite direction of a hivemind with a loud set of background assumptions, e.g. everyone putting emojis in their username to mark their alliances. But sometimes it is helpful to establish what does and doesn't go without saying.

For about a yearish I've been with varying frequency writing down things I've learned in a notebook. Partly this is so that I go "ugh I haven't even learned anything today, lemme go and meet my Learning Quota", which I find helpful (I don't think I'm goodharting it too much to be useful). Entries range from "somewhat neat theorem and proof in more detail than I should've written" to "high level overview of bigger subjects", or "list of keywords that I learned exist". For example, recently I learned that sonic black holes which trap phonons (aka lattice vibrations) are a thing, and the quantum theory predicts they should emit sonic hawking radiation, and unlike hawking radiation we've actually observed this directly by making a sonic black hole in the lab. A side effect is that... wow I learn more than I think I do. My retention isn't perfect but it's a lot better than what I would've expected had I known I learn that much. Every month there's usually a handful of things I've learned that register as "Really really cool/important/fundamental".

Thought in progress: epistemic humility is not a substitute for actual humility (or professed humility). You only get to cry wolf once, but you can probably warn about potential wolves several times—so long as you don't burn goodwill on an incorrect or overconfident prediction. I think epistemic humility helps to increase trust and confidence in EA/Less Wrong-type spaces, but I think professed humility is far more helpful when it comes to public-facing AI comms, particularly as scenarios get more intense and specific (e.g. prefacing AI doom predictions with a decent amount of throat-clearing beforehand commensurate with the intensity and specificity of the forecast). For example, I think that AI 2027 might have been better received if the authors had spent less time trying to convince the readers of their credibility at the beginning and spent more time saying something along the lines of "we know this sounds crazy and are well aware of how sci-fi the scenario seems". (I'm not a huge fan of lampshading in fiction, but IRL, I think you do need to display self-awareness of outlandishness in order to be taken seriously, particularly if what you're predicting sounds insane to the average person.) Of course, there are huge diminishing returns on this: the more throat-clearing you do, the less confident you seem. And throat-clearing should probably be saved for public-facing comms, because actual technical work seems to require people who are confident in their beliefs even when they are outlandish (as proven by the outlandish explosion of AI progress recently). Still, I think that the AI safety community at large has a worse reputation than they deserve, and I think part of that is due to the appearance of overconfidence. This problem seems simple, tractable, and important.

There is something weird about LLM-produced text. It seems to be very often the case that if I'm trying to read a long text that has been produced primarily by an LLM, I notice that I find it difficult to pay attention to the text. Even if there's apparently semantically rich content, I notice that I'm not even trying to decode it. Robert wrote [...] It's kind of similar to the times when your attention wanders away during reading, and then you realize that you were scanning/semi-reading the last page with your eyes, but the information mostly didn't make it past the visual cortex, and you failed to construct the coherent scene that the book was trying to put in your mind. In the case of LLMs, a milder version of this seems to be the default. This is much more (and much more often) the case if an LLM is tasked to produce something "original-ish" (an essay on topic X, say), rather than just give you the facts on some topic. What's going on?

Last year, METR used linear extrapolation on country-level data to infer that AI world takeover would ~never happen. However, reviewers suggested that a sigmoid is more appropriate because most technologies follow S-curves. I just ran this analysis and it's much more concerning, predicting an AI world takeover in early 2027, and alarmingly, a second AI takeover around 2029. Here are the main differences in the improved analysis: * The original analysis assumed that the equation for AI takeover vs year would be axis-aligned, but this is an arbitrary basis. We now let the angle of the sigmoid fit the data as appropriate, meaning the following data model: Interestingly, this results in a Z-curve (A < 0) rather than an S-curve. This is consistent with advanced AI being a substitute rather than a complement to labor. * We now incorporate our priors. Even though we only have hard data since 2023, it's likely that the number of AI takeovers was lower pre-ChatGPT. So we added the inferred point (Nov 30 2022, -0.03). The rotated sigmoid still gets R^2 = 1.000 and now looks much better on BIC than exponential and step-function models, both of which have log-likelihood of minus infinity on negative points. * This new graph uses only aggregate AI world takeover data, which we're much more confident in than our data from individual countries. The most important next step is confirming that the # of AI takeovers was large negative in the past. If so, we should start preparing for AI takeover as a recurring rather than one-time event, which could have significant implications for movement strategy. For example, we may need to pivot into something more like seasonal preparedness, similar to how society handles flu outbreaks or daylight savings time.