Of course. Particularly if all you are interested in is a filter to improve water taste, you can purchase a standard GAC filter and place it to filter the cold water supply to your faucet. Unless you use an inordinately large amount of water, changing the filter every six months should be adequate on a city water supply.

what about the hot water? do i just have the cold water in that faucet then?

Filters aren't designed for hot water.

The water from the faucet will be filtered when you run cold water, not filtered when you run hot. The only good way to have filtered hot and cold would be to filter all of the water entering your house.

i thought i need a drinking water filter even if i filter all of the water entering the house?