Winnie the Pooh series by AA Milne and Ernest H. Shepard (Winnie-The-Pooh, The House at Pooh Corner, When We Were Very Young, Now We Are Six). Accept no substitutes for the original stories!
Beatrix Potter series (Peter Rabbit, Benjamin Bunny, etc,)
The Little House by Virginia Burton
The Story About Ping by Marjorie Flack
The Little Engine that Could by Watty Piper
Blueberries for Sal by Robert McCloskey
Make Way For Ducklings by Robert McCloskey
One Morning in Maine by Robert McCloskey
Ferdinand by Munro Leaf
Ox-Cart Man by Barbara Cooney
Stone Soup and other folk tale retellings by Marcia Brown
Miss Rumphius by Barbara Cooney (CM idea of duty in bringing beauty to the world)
The Story of Little Babaji by Helen Bannerman or other retelling of the Sambo story with more appropriate illustrations
Brer Rabbit books by Joel Chandler Harris, read the online etext
Poems and Prayers for the Very Young by Martha Alexander
A Child's Garden of Verses by Robert Louis Stevenson (illustrated by Eulalie, Jessie Wilcox Smith, or Alice and Martin Provenson)
A good collection including classic stories and folktales such as The Little Red Hen, The Gingerbread Man, Goldilocks and the Three Bears, The Three Billy Goats Gruff. (Joseph Jacob's versions online here and here)
A good collection of Aesop's Fables, such as the one illustrated by Milo Winter
A nice Mother Goose collection, such as The Real Mother Goose illustrated by Blanche Fisher Wright Mama Goose, a collection by Edelen Wille, or versions such as those by Rojankovsky, Marguerite De Angeli or Tasha Tudor, or Lavender's Blue, a collection of nursery rhymes compiled by Kathleen Lines, illustrated by Harold Jones
Illustrated classic poetry such as Poems for Young Children compiled by Caroline Royds
A good collection of classic children's poetry such as A Child's Book of Poems by Gyo Fujikawa; The Golden Books Family Treasury of Poetry selected by Louis Untermeyer; The Oxford Book of Children's Verse edited by Peter Opie