Nationally, median household income rose by 4% between 2022 and 2023, per the latest U.S. Census Bureau data. Unfortunately, though, incomes weren't growing everywhere.

The median household income in Marysville, Washington, a city just outside of Seattle, dropped by nearly 18% during that time — from $104,433 in 2022 to $85,708 in 2023 — according to a SmartAsset analysis of Census Bureau data. Marysville residents experienced the largest annual decline in median income out of over 600 U.S. cities with populations of at least 65,000, SmartAsset reports.

In three other cities — Champaign, Illinois; Madera, California; and Baton Rouge, Louisiana — median incomes dropped by at least 17%. 

Here are the cities that saw the fastest declines in median household income between 2022 and 2023, according to SmartAsset:

Why incomes are falling in Marysville

It's likely a number of high-earners left Marysville, which contributed to the decline in median household income. The share of local households earning at least $100,000 fell from 54% in 2022 to about 41% in 2023, SmartAsset finds. The share of households earning at least $200,000 grew, but only slightly.

"High-earners generally have more means to leave and seek out opportunities elsewhere," Jaclyn DeJohn, director of economic analysis at SmartAsset, says. "This can cause major changes in the earnings of a particular community."

Despite the drop in earnings in Marysville, the median income of $85,708 there is still above the national median of $80,610.

An individual earning that much in Marysville could feasibly live well, given that it takes a $61,545 annual salary for an individual without kids to afford basics, including rent, food, health care and transportation, in Snohomish County, Washington, where Marysville is located, according to the Massachusetts Institute of Technology's living wage calculator.

In the state of Washington overall, the median household income in 2023 was $93,440, bolstered by higher-earning cities like Seattle and Bellevue, where median incomes are over $100,000.

Incomes are shrinking in metros where pay already lags the rest of the U.S.

Though median incomes fell at similar rates in Champaign, Madera and Baton Rouge, residents there already lagged the rest of the country in pay. In each of those places, the median household income was below $70,000 in 2022 and fell below $60,000 in 2023.

Part of that may be attributable to age, DeJohn says. In Champaign and Baton Rouge, for example, the working age population — those ages 20 to 64 — fell by 1 to 3 percentage points between 2022 and 2023, according to Census Bureau data.

"This could indicate that people may be dropping out of the workforce, or taking smaller incomes for more flexibility, in order to take care of children [or] growing families," she says. "Similarly, workers may age out into retirement, and many retirees bring in a lower income than when they were in their prime working years."

Residents in the Midwest and South broadly earn less than other regions in the country, and Louisiana has the second-lowest median household income at $57,650.

While Illinois and California tend to be higher-earning states, Champaign and Madera are pretty far from major cities in those states where high incomes tend to be more concentrated. Illinois, for example, has a median household income of $87,820. But that's brought up by Chicago and its metro area, where the median income is $74,474, nearly $30,000 more than what folks earn in Champaign.

Want to make extra money outside of your day job? Sign up for CNBC's online course How to Earn Passive Income Online to learn about common passive income streams, tips to get started and real-life success stories.

Plus, sign up for CNBC Make It's newsletter to get tips and tricks for success at work, with money and in life.