The story British art tells about itself tends to run through native genius: Turner's light, Constable's fields, the Bloomsbury painters at their easels. It is a comforting story and an incomplete one. A great deal of what counts as modern British art was made by people who arrived from somewhere else, often with nothing and frequently unwelcome. Poles, Russians, Germans and Austrians, Jewish families fleeing first the pogroms and then a genocide, carried European ideas into London studios and changed the course of the country's painting. One small institution has spent more than a century keeping the record straight. The Ben Uri Gallery and Museum, founded in the East End in 1915, exists to document what it calls the immigrant contribution to British visual culture, and the collection it has assembled amounts to an argument: that the foreign-born artist is central to modern British art rather than a footnote to it.

A society founded on pennies

Ben Uri began on 1 July 1915, in Gradel's restaurant in Whitechapel, in the crowded heart of London's Jewish East End. Its founder was Lazar Berson, a Russian émigré artist, and its first name was a mouthful: the Jewish National Decorative Art Association (London), Ben Ouri. The shorter form stuck. It comes from Bezalel, son of Uri, the biblical artisan charged in the Book of Exodus with building the Tabernacle, a fitting patron for a society of craftsmen and painters.

The society answered a plain need. Immigrant Jewish artists were largely shut out of the mainstream art world, unable to gain entry to the established societies and galleries that decided who was seen. Ben Uri gave them a place to exhibit and a collection that would hold their work. What makes the founding remarkable is who paid for it. The money came not from the wealthy Anglo-Jewish philanthropists of the day but from the immigrants themselves, in pennies rather than pounds. A community with very little built an institution to keep its own art from vanishing.

That instinct proved durable. Ben Uri lost its dedicated gallery in 1995 when the synagogue building that housed it was sold, relaunched in 2001, and moved to St John's Wood in 2002. Its mission has widened well beyond its origins. Today it describes itself as a research resource devoted to the refugee and immigrant contribution to British art since 1900, with a collection spanning some 440 artists from 44 countries.

The Whitechapel Boys

The world Ben Uri was born into produced the first great cohort of immigrant-descended British modernists. Around the Whitechapel Library and the Whitechapel Art Gallery, a loose circle of young Jewish artists and writers gathered in the years before the First World War. Posterity has called them the Whitechapel Boys. They were the children of immigrants, some born in England to families that had fled the pogroms of the Russian Empire, and they were poor. They were also ambitious in a way that reshaped British art.

The painters David Bomberg and Mark Gertler stood at the centre of the group, alongside the poet Isaac Rosenberg, the painter Jacob Kramer and, as the one woman usually named among them, Clara Birnberg. Gertler was born in Spitalfields in 1891, the youngest child of Polish-Jewish immigrants, and grew up in real poverty. A grant from the Jewish Education Aid Society sent him to the Slade School of Fine Art, where his contemporaries included Stanley Spencer and Paul Nash. His painting "Merry-Go-Round" of 1916, now at Tate, turns a fairground carousel into a scream of mechanised futility, one of the fiercest anti-war images of its century. Gertler was a conscientious objector, and the picture shows it. D. H. Lawrence called it the best modern picture he had seen.

Rosenberg, remembered now mainly as one of the finest poets of the Great War, trained as a painter at the Slade before he was killed on the Western Front in 1918. The circle did not survive the war intact, but its example did. These were the artists who pushed continental modernism into modern British art from below, rising out of the immigrant East End rather than the drawing rooms of Bloomsbury.

Bomberg's long shadow

David Bomberg is the hinge of the whole story. The son of Polish-Jewish immigrants, raised in poverty in Whitechapel, he trained at the Slade and emerged just before the First World War as one of the most radical painters in Britain. Works such as "The Mud Bath" and "In the Hold", both from around 1914, break the human figure into hard, faceted planes of colour and stand with the Vorticists at the edge of British abstraction. Few English painters of the moment were bolder.

His later importance was as a teacher. From 1945 Bomberg taught at the Borough Polytechnic in south London, where his classes drew a striking set of students and gave rise to the Borough Group in 1946. Two of those students matter enormously to what came next: Frank Auerbach and Leon Kossoff, the latter the London-born son of Russian-Jewish immigrants. Through Bomberg, the immigrant modernism of the Whitechapel Boys passed directly into the post-war generation that critics would later group as the School of London. The line from the pre-1914 East End to the paint-laden canvases of the 1950s runs through one man's evening classes.

The exodus of the 1930s

The second great wave came under duress. As the Nazi regime tightened its grip on Germany and then Austria, artists judged racially or politically undesirable began to leave, and Britain became a refuge for many of them. More than 300 visual artists fled Nazi-dominated Europe for Britain. They reached a country that offered safety without much warmth, and often without work.

Some names in that exodus are now fixed in the canon. Piet Mondrian, Naum Gabo, László Moholy-Nagy and Walter Gropius all spent time in Britain during the 1930s, part of a modernist influx that briefly made London a serious centre of the avant-garde. Several treated the country as a way-station and moved on, a number of them to the United States. Others stayed. The Austrian expressionist Oskar Kokoschka, already famous, reached Britain in 1938 by way of Prague and became a loud anti-Nazi voice in exile, serving as honorary president of the Free German League of Culture. The Polish painter Josef Herman fled Warsaw and reached Glasgow in 1940, later making the Welsh mining valleys his great subject.

Two of the refugees were children. Lucian Freud, born in Berlin in 1922, the grandson of Sigmund Freud, came to London with his family in 1933 as the Nazis rose, and became a British subject in 1939. Frank Auerbach, born in Berlin in 1931, was sent to England alone in 1939. His parents stayed behind and were murdered at Auschwitz in 1942. Both boys grew up to become defining figures of British painting. Help for such arrivals came largely from private hands. The Artists' Refugee Committee, set up by figures including Roland Penrose and Margaret Gardiner, did what the government would not, finding sponsors and guarantees for artists in danger.

Interned as enemy aliens

Britain's welcome had a cruel turn in it. At the outbreak of war in September 1939 some 74,000 German and Austrian nationals in Britain were classed as enemy aliens, most of them refugees from the very regime the country was about to fight. In the spring and summer of 1940, with invasion feared after the fall of France, the government interned them in large numbers. Thousands were shipped to camps, many to the Isle of Man, and held behind wire as suspected enemies despite being, in most cases, Jewish or anti-Nazi and no threat at all.

The absurdity produced one of the strangest episodes in twentieth-century art. Hutchinson Camp, in Douglas on the Isle of Man, held so many painters, sculptors, art historians and writers that it became known as the artists' camp. Among the internees was Kurt Schwitters, the German Dada pioneer and inventor of Merz, who had already fled to Norway and then to Britain. In the camp he set up a studio and made portraits of his fellow prisoners. Released in 1941, Schwitters eventually settled in the Lake District, where he began a final version of his life's project, the immersive construction he called the Merzbau, in a barn near Ambleside with support from the Museum of Modern Art in New York. He died in Kendal in January 1948, the day after he was granted British citizenship.

What the émigrés built

The long-term result is visible on the walls of every major British collection. Auerbach and Kossoff, taught by Bomberg, worked thick, laboured surfaces into a distinctly British kind of figurative painting. Together with Freud they became central to the School of London, arguably the most important British art of the post-war decades. Many émigrés shaped the country less through their own canvases than through teaching, carrying a continental seriousness about colour and feeling into British art schools. Painters such as Martin Bloch and Georg Mayer-Marton trained a generation of British students in ideas they had brought out of Germany and central Europe.

This is the case Ben Uri has spent more than a century assembling, artist by artist. Its collection now records the work of hundreds of immigrant and refugee artists across British art since 1900, and read together it tells a story the standard national narrative tends to skip. The people who fled here, or whose parents did, were not guests at the edge of British culture. They were among its makers. To walk through the émigré contribution is to see that a great deal of what the world thinks of as quintessentially British culture was dreamed up by outsiders who had every reason to give up and did not.