It is likely that you are forming a soft scale in the heat exchange tubes that is being removed during those times when more water volume is running through them. Are you performing a periodic descale? Do you know your water conditions (analysis)? Have you called Rheem to consult with their techs?

are you on a well?

Thanks for the replies. I am on city water that is very soft. The heater is 3 years old. How does one descale a tankless heater?

If your water is already very soft, it becomes less likely that you have scale and more likely that you have particulates coming from the city water or corrosion products. Take a look at your city's quarterly or annual water quality report. Many cities post theirs online, but if not you are entitled to a copy. Post that report and I'll take a look to see if anything looks like a likely culprit.

Do you notice any particulates in your water? Rusty-looking stains on any appliances or fixtures?

No other problems with water, only when second faucet in use.

Flushing the tankless involves running vinegar or a mild acidic cleaner through the piping/heater to remove any scale. I'm not sure about yours, but Noritz has a valve kit that lets you do this.

It's basically two three-way valves that you install in the pipes in/out of the heater. The third port is capped, and you can flip the valves to cut off normal flow and supply/return water from these ports. Then you hook up tubing to a small pump (probably a little submersible one)in a bucket with a gallon or so of vinegar. Turn the pump on and recirculate.

All that said, and we didn't get the flushing valves installed when we put in our heater. It's on my long-term list to add them. Fortunately, our municipal water is super soft.

One other thought- do you have any old pipes at your house? If you still had any galvanized pipe in an old house, it would tend to flush iron sediment when you crank up the flow. Our house originally had galvanized water piping that we replaced over the years.

I'm taking issue with the title... "rusty water from tankless heater".

Keep in mind that virtually all tankless heaters I'm aware of use copper tubing in the heat exchanger. There's no iron pipe - and thus no rust coming from the tankless heater.

What kind of piping is in the rest of the house? Did someone use black pipe - gas line -- for water pipe?

Any heating device will cause lime, if present in the water, to collect on the hot surfaces. Descaling the heater using vinegar (usually is what's recommended by manufacturer & because it's food grade) every 2-3 years is reasonable.

That said, lime isn't the color of rust - it's grey- white.

I would look other places for the source of your rusty color.