At least 17 people have been rescued by a merchant ship that is in the area.
But a spokeswoman for the Australian Maritime Safety Authority (AMSA) said there was grave concern for the safety of any other people still in the water.
The Cocos Islands – an Australian ocean territory – lie roughly halfway between Australia and Sri Lanka.
Amsa said it had received a distress signal on Sunday reporting that a ship had a hole in its hull and was taking on water in rough seas about 350 nautical miles (650km) north-west of the island group.
The chief of the Australian Defence Force, Air Chief Marshal Angus Houston, told reporters that a Taiwanese trawler had been first to get to the scene late on Sunday night
"When the first ship got there this vessel was still intact," he said.
"Somehow or other during the process of the interaction between the ship and the trawler, and also the stricken vessel, there’s been a capsize and people have ended up in the water," he added.
By the time the merchant ship, LNG Pioneer, had arrived, the boat had already sunk, he said.
The LNG Pioneer and the trawler are continuing to search for more survivors. An Australian military aircraft has been sent to the area.
"The LNG Pioneer is a large vessel and they are trying their best with life rafts and their life boat to recover those in the water," Amsa spokeswoman Rhianne Robson told the Australian Broadcasting Corporation.
(For updates you can share with your friends, follow TNN on Facebook and Twitter )