In 1682, Pierre De Garmo signed a contract for the purchase of Samuel Wilson's house and lot in Albany. He was identified as "a French vagabond" and was known as "Viela Roy" or "Villeroy." By that time, he had married Albany native Catharina Vanderheyden and had begun a family. He was the father of at least ten children, most of whom were baptized in the Albany Dutch church - where his wife was a member.

Trading for furs from a base of operations near Saratoga, the Frenchman Villeroy experienced more than his share of difficulties and appeared frequently before the Albany court. Recognized as a "merchant," he was fined for illegal trading, accused of taking "a considerable sum of money," was the subject of a complaint from the governor of Canada, later was under suspicion of being a French agent and, for a time, was held in custody. Many Frenchmen were similarly suspected in seventeenth century Albany.

Villeroy weathered these difficulties. By the end of the century, he was known as Pieter De Garmo and had established his family in Albany. Identified as of French ethnicity in a home with his wife and seven children on the census of householders in 1697, De Garmo could not take the loyality oath tendered to all residents in 1699 because he was a "papist." He did, however, join Albany's "loyal protestants" in a petition to the royal governor in 1701. Settling into a modest home near the northern edge of the Albany stockade, De Garmo bolstered sagging trade opportunities by working as a laborer.

In 1719, he purchased some land along Foxes Creek and, a few months later, an additional lot at the foot of Gallows Hill. He seems to have moved his family across town for, in 1720, his name appeared on a list of qualified Albany voters living in the first ward.

Patriarch of one of colonial Albany's first French families, Pierre De Garmo died in March 1741 and was buried from the Albany Dutch church. The children of Pierre and Catharina established themselves in Albany and throughout the region. The De Garmo family maintained a small but steady presence in Albany throughout the eighteenth century.