General Notes

BB: Still only one Charles Bruster in these Census in Philly. (1820, 1830, 1840)
1820 United States Federal Census about Charles Bruster
Name:Charles Bruster
Home in 1820 (City, County, State):Philadelphia Middle Ward, Philadelphia, Pennsylvania
Enumeration Date:August 7, 1820
Free White Persons - Males - 16 thru 25:1
Free White Persons - Males - 26 thru 44:1
Free White Persons - Females - Under 10:1
Free White Persons - Females - 16 thru 25:3
Free Colored Persons - Females - 14 thru 25:1
Number of Persons - Engaged in Commerce:1
Free White Persons - Under 16:1
Free White Persons - Over 25:1
Total Free White Persons:6
Total Free Colored Persons:1
Total All Persons - White, Slaves, Colored, Other:  7

DB: The Iowa Charles (born 1813) possibly never headed a household in Philadelphia, and left no trail there.  The Madisonian article claims he moved to Iowa in 1844, after spending some time in Indiana. His first wife was 10 years younger, so if he did marry in Philadelphia, they may not have stayed there long.
By the way, in 1820 there are both a Chas Bruster and a Charles Bruster in the Middle Ward!  Very similar households, but less females under Chas.
The 1830 census shows one male aged 15-19 living in Charles's household, so he could be the younger Charles. The 1820 census does not show any male aged under 10, which is consistent with young Charles arriving in 1826. This could be significant unless it already is a boarding house.
However, the 1820 households are too substantial to support the idea that they were run by 'my' Charles.  The oldest male is aged 26-44, but there is also a male aged 16-25. 

BB: It would be easy to say that your Charles was named after an uncle Charles. 
AND, We need to remember that John Bruster was buried in 1829 by the 1st Reformed Presbyterian Church where both a Charles and William Bruster were members.  This certainly ties him to these two people.  
AND the 1st Reformed church is in this North Ward.