On reaching Sydney in September, Governor Brisbane granted him 2,000 acres at the place called Aborigines Wallerawang. This land had been by-passed on the way to Bathurst by the early settlers and in 1837, he applied for more land between the Wolgan Valley and Bathurst. He started 1800 cross-bred sheep, 64 merinos, 312 head of cattle and 15 horses.

By the end of 1840, James Walker held an immense tract of country under licence and was running 1487 cattle and 20,534 sheep. In 1854 he was the holder of 16 stations with an area of 470,000 acres. James Walker returned to Scotland on many occassions while Andrew Brown joined with David Archer, a young man of 17. One of nine, eventually they all came to Australia. It was during this time too, that Charles Darwin paid a visit to the Wallerawang homestead, where David Archer took him on an unsuccessful kangaroo hunt. He did shoot a platypus though for Darwin to study.

Walker's first born was named Allison, born in Wallerawang in 1834, his second, Wilhelmina, was born in London. On returning to Australia once again, Walker and his wife Robina Ramsay Walker (his cousin) had two more children; Archibald James in 1841 and Georgina Lyon Wolgan in 1843.

The male line of the Walker family ceased when his son, Archibald, died in 1858 of a fever in Glasgow. Allison married a New Zealander and moved to London after selling her share of the estate, later dying in 1912, aged 78. Wilhelmina (possibly deformed) married a cousin and died at childbirth aged 28 in 1854. Georgina married Edwin Barton. He was a surveyor who was employed to map out the route for the railway to the west.

James Walker himself died in 1856, aged 71, and in 1866 his widow still held licences for 15 stations, she died in 1867. The last chapter of this family is a sad one. Georgina's son James had three children. The eldest two were unmarried and lived together at Wallerawang. They were both murdered in 1848, shot dead by a young farm hand.