Why?

Our little gold fish seems sick. It stays at the bottom of the fish tank a lot. I noticed that since we started having gold fish and a few variety of fishes in our house (we had bought about 6 more varieties in the last few months) our gold fishes just die off from unknown cause. Why did they die one by one. What is the root of the problem? Was it due to the fact that I kept on changing the tank's water 2x a week. Meaning we haven't got a filter to do it.

At first I thought it was the water because I got it from our faucet. You have to leave it in a container without cover 2 days to let the chlorine escape.. Another is I thought the janitor fish kept nudging my calico (another variety of fish with bulging eyes and usually black in color). His scales were removed and he became more and more skinny and died. Thirdly I thought it was the newly purchased food for them, because it was so hard that the moment they swallow it they spit it out.

Anyway, I thought the filter wasn't essential (thought it just now huh). Well our fishes were stressed and now I just bought them a filter and they seemed relax now. Thank goodness for that c: Also less stressful for me. I don't have to change the water of the tank so regularly c: