That depends greatly on your ISP, the type of connection (cable, dsl, ftth), and your router's specifications. In general, it is not the raw upload speed but the number of connections that will affect your download speed. For instance, uploading to an FTP at full speed should not greatly affect your download speed, but uploading small amounts to 20-30 peers at once will create a lot of overhead and your network card may slow as a result.
But there are many ISPs that use traffic shaping rules which will slow you down if you upload at max capacity for a long period of time (15min - 1hr+) -- and they tend to target the bittorrent protocol. Then again, performance can still vary from one torrent to another.
Basically, if you suspect your download is unusually slow, you may try to limit your upload. But there's no guarantee it will help speed up the download, and there's no need to limit the upload speed permanently.