Shortly after Jimmy was born, his father abandoned the family. By the time he was 13 and living in the Tomball area of Houston, his mother kicked him out of the house. As he recalled in a 2011 interview with ESPN, her last words to Jimmy were, "I don't like the look of you. You gotta go." And then he was out on the streets.

With nowhere to go, Jimmy stayed with various friends for a few weeks at a time before moving on to a new place. By his senior year of high school, Jimmy met a ninth-grader named Jordan Leslie, and the two became fast friends.

Jordan challenged Jimmy to a three-point shooting contest; after the competition, the future NFL wide receiver invited Jimmy over to his house to play video games and stay the night. Their bond grew, so much so that Jordan took Jimmy in. After a few weeks, Jordan's mother, Michelle Lambert, discovered that Jimmy had been living with them.

ESPN noted that Michelle was hesitant at first — she had four kids of her own from her late first husband, and her new husband brought three children of his own with him. Money was tight, but word on the street was that the basketball star was in trouble. Michelle's husband told the kids that Jimmy could stay, but only for one or two nights at the time.

However, each night that Jimmy went to stay at the Lamberts, a different child would say, "Tonight's my night to have Jimmy stay." Eventually, after a few months, Michelle told Jimmy he could stay for good.