Loading video...

Video Failed to Load

Go Home

Predicting the next word "only" is sufficient for language models to learn a large body of knowledge that enables then to code, answer questions, understand many topics, chat, and so on. This is clear to many researchers now, and there are nice tutorials on why this works by Ilya...

130,179 views • 2 years ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

They did not take cursive from the schools because children no longer needed it. They took it because of what it was quietly building in them. Consider what the exercise actually is. A child, six years old, is handed a pen and asked to draw a single unbroken line that becomes a word. The wrist must float. The fingers must hold a living pressure, never quite the same twice, always correcting. The eye must follow the ink forward and trust the hand to finish what it has begun. There is no lifting, no stopping, no starting over mid-word. The loop must close. The ascender must rise and return. The sentence must travel from one margin to the other as a single continuous gesture, and at the end of it the hand must still be steady. Twelve years of this. Every day. Ten thousand small acts of sustained, self-correcting attention, carried out below the level of conscious thought, until the motion belongs to the body and the body belongs to the motion. This is not penmanship. It is the slow construction of an interior form. The hand that has learned to carry a line without breaking it is the hand of a mind that has learned to carry a thought without breaking it. The two are not metaphors for one another. They are the same faculty, trained in the same child, by the same daily discipline. Continuity of the stroke becomes continuity of the reasoning. The patience of the loop becomes the patience of the argument. The commitment to finish a word one has started becomes the commitment to finish a sentence, a paragraph, a life's idea, without reaching for the nearest distraction halfway through. Print is a different creature entirely. Print lifts. Print stops. Print assembles a word out of separate, stamped, interchangeable pieces, each one beginning and ending in isolation. A mind raised only on print learns to think the way print is made, in discrete tokens, in replaceable units, in fragments that can be recombined by any outside hand without the owner noticing the substitution. It is precisely the shape of thought a language model produces. It is precisely the shape of thought a language model can steer. Cursive is kata. This is the whole of it. A form repeated daily, for years, not for the sake of the form but for what the repetition lays down in the practitioner beneath the form. The swordsman does not train kata so that one day he may fight in kata. He trains it so that when the moment comes and there is no time to think, the movement is already inside him, older and deeper than thought, and it rises on its own. Cursive was the kata of the literate mind, the daily quiet drilling of continuity, of patience, of a line held steady under the long pressure of its own length. And the signature it produced at the end, that small flourished mark unique to a single human being on earth, was only the outward proof of an inward form no machine and no other hand could ever reproduce. Take the kata away and the practitioner is left with vocabulary in place of faculty. He can recognise a whole thought when he encounters one. He cannot carry one himself. He can admire a finished argument. He cannot sustain one long enough to close its loop. He begins books he does not finish, sentences he does not end, ideas he abandons the moment the screen in his palm offers him a brighter one. And when the machine begins feeding him tokens in the exact shape his schooling taught him to receive, he meets it with no interior resistance at all, because no interior form was ever built in him to push back with. They removed it quietly, across a generation, and they removed it in the last years before the machines arrived. Twelve years of daily practice in unbroken, embodied, self-authored thought, gone from the curriculum of almost every child in the Western world, just as the instruments designed to complete their sentences for them came online. The hand forgets. The mind, having never been taught the kata, forgets a thing it never knew it had. That is what cursive was. That is what was taken. And that is why the thought of anyone who still writes by hand, in long unlifted lines, remains, quietly, stubbornly, and without their ever needing to announce it, their own. Now the question stands open. What else has been banned, phased out, quietly retired from the curriculum and from common life over these same decades, under the same soft excuses? Mental arithmetic. Memorisation of poetry. Latin. Logic as a formal subject. Map reading. Knot work. The keeping of a commonplace book. The reading aloud of long passages in class. Singing in parts. What was each of those actually building in the child, beneath the surface of the lesson, and whose interest was served by its disappearance?

SiriusB

440,735 views • 2 months ago

Today I was visiting the exceptionally beautiful Plantin-Moretus Museum in Antwerp, Belgium (one of the only museums in the world that is itself listed as UNESCO World Heritage), which is one of the oldest printing shops in Europe, with the oldest surviving printing presses in the world. I stumbled upon an old 16th century atlas - written in Old French - and I was pretty amused to read their understanding of China at the time, which was surprisingly accurate, maybe even more than today's! A translation of some of the most interesting passages: - They call it "China" in French (it's now called "Chine") and they write that the locals call it "Tangis", which probably refers to the Tang dynasty but which is strange given that by the 16th century the dynasty had already ended for about 600 years - They write that to its North China is bordered by "tartares" (which I guess means Mongols) whom they describe as "very warlike people from whom it is separated by a wall made by hand" - The Chinese work ethic was already legendary: "those who live there are not at all lazy but devoted to labor and work, because it is there a shameful thing to be idle" - They share a number which must have seemed astonishing at the time: "in the city of Canton, one of the smallest in the entire country, some ten or twelve thousand ducks are eaten daily at table". And then they marvel during a good proportion of the text about the abundance of food in the country, which probably made a big impression on travelers at the time. - They write that "there are in this kingdom two hundred and forty famous cities, whose names end in this syllable FU which means a city: like Cantonfu, Panquifu: the small towns, which are in great number, end in CHEU [undoubtedly refers to "zhou"]. There are infinite villages, heavily populated, because of the continuous agriculture." - China's infrastructure and engineering capabilities were also already legendary at the time: "The city gates have entrances magnificently and marvelously well made, the streets are made level, not sloping this way or that, but following their straight line. They are so wide that ten or fifteen men on horseback can march abreast and are everywhere marked and separated by triumphal arches that marvelously ornament the cities. Portuguese say they saw in the city of Fuchco [probably Fuzhou] a tower set on forty solid marble pillars, the height of which was forty palms (masonry measure) and the width twelve: that this work is so grand, so exquisitely made, so beautiful to see, so sumptuous and so pleasing that it far surpasses all the magnificent buildings of all Europe." - Already at the time, China was very wary of safeguarding its sovereignty: ""[The Chinese] rarely or never leave their country and do not easily let foreigners enter it, especially into the interior of the province, unless they first have safe conduct from the king." - On moral and cultural habits: "They put adulterers to death. There are no brothels in the cities, all manner of prostitutes being sent to the suburbs. They celebrate their weddings at the time of the new moon and around the month of March which is their first day of the new year, and they make these celebrations, like us, very magnificently. They show themselves valiant in banquets and entertainments, in which they owe nothing to the Flemings or the Germans. They eat at tables like us in Europe, on chairs or on benches, and not on the ground as other peoples of Asia do." - On justice: "Bandits and murderers are kept in perpetual prison. Theft, which is a very odious crime, is punished by whip strokes in this manner: they put a man belly down, tie his hands behind him, striking him on the fleshy part of the legs with a whip made of reeds or canes." - On China's naval capabilities at the time: "This kingdom has an infinite number of ships, galleys and vessels of all sorts, with which they cross the seas and rivers. So much so that when they want to show through vainglory the power of their king, they are accustomed to say in a common proverb that he can make a bridge of ships joined together, which can reach and extend from China to Malacca, which is a distance of five hundred leagues and more." - On the emperor and China not being warlike (already back then): "All this region is subject to a single king, like a monarch; whom they call lord of the world and son of the sun. He holds court at Paquin [Beijing], which is a city toward Tartary. He never leaves it, except in time of war. It is said that when he makes war on the Tartars he leads an army of three hundred thousand soldiers and two hundred thousand horses, although it is also said that this nation is not very warlike. This king has under him fifteen very large provinces, which they call governments, and he alone surpasses in power all the other neighboring princes of Asia; and his annual revenues exceed all the riches of Europe. Antonio Pigafetta [the chronicler of Magellan's voyage] calls this king the most powerful of all the universal earth and says that the royal city is fortified and ramparted with seven walls, having ten thousand soldiers for the guard, and that the king commands seventy other crowns of the royal diadem [likely refering to the tributary state system]." Reading these passages, it seems that the further we've come in our ability to know China, the more obscured our vision seems to have become. These 16th century observers, working with fragments brought back by explorers, merchants and missionaries, managed to capture the essential - the industriousness, the engineering mastery, the administrative sophistication, the careful sovereignty. They approached their subject with the humility of the genuinely curious. They had no framework to force China into, no predetermined narrative to fulfill. They simply watched, counted ducks in Canton, measured city walls, and wrote it down. Their errors were errors of transmission - a dynasty name lingering centuries past its time, numbers perhaps inflated through retelling - but the spirit was one of simply describing unknown territory, not to convince anyone of anything. Today however, drowning in information, we're somehow seeing less of what's there and more of what we expect to find. Each observation must fit into existing narratives, serve predetermined conclusions, advance familiar arguments. So much so that we must ask ourselves: have we actually moved backward from those 16th chroniclers? Maybe we need to re-learn to approach China - and others in general - like those old cartographers, pen in hand, ready to be surprised? What might we discover if we stopped explaining and started counting ducks again?

Arnaud Bertrand

19,380 views • 10 months ago