A tour guide sits atop a double-decker bus in Toronto’s downtown core, clutching his microphone in the frigid wind. Except for one brave soul, the seats around him are empty.

Even down below, a British pair pull their toques on and complain about the chill. Another couple, from New Zealand, are still adjusting after a family visit in Arizona, where the temperature was in the mid-30s.

“We’ve gone from the sublime to the ridiculous,” Annette Sutherland said, smiling.

There is not a U.S. tourist to be found on the bus, but that’s not unusual, the guide, John Lebow, said. “With Americans … it was already on the slow side. There seems to be a bit less.”

The Sutherlands’ son lives in the United States and travelled here with them – but only to attend a conference of the American Thoracic Society. Many Americans have stopped coming here as tourists, Sam Ghoby of Toronto Tours said. “The main thing, right now, is convention clients.”

In the bustling lobby of the Delta Chelsea hotel, there is no sign that tourism is in decline. But looks – and reservation books – can be deceiving, concierge Neyba Cardoza said.

“The reason it’s so full right now is because of conferences,” she said. “Before, we were getting a lot of Americans, and this year, it’s down.”

One thing that keeps them coming is baseball, according to a host at the Loose Moose restaurant on Front Street near the Rogers Centre. “We get lots of Japanese tourists, lots of Americans,” Beau Tennant said.

A couple of blocks away in the theatre district, it’s a different story. Restaurant Le Saint Tropez has lost at least a quarter of its U.S. clientele, manager Victor Magellan estimates.

“Our first hit was SARS,” he said, followed by the flagging U.S. dollar and the law requiring Americans to have passports when crossing the border. “It definitely affects us. They’re not coming in the same amounts.”

Still, Toronto’s tourism industry saw it coming, according to Ms. Cardoza. “We all expected this,” she said. “With the economy, we all expected tourism to go down.”