NFL Network Insider Ian Rapoport and NFL Network's Mike Garafolo first reported the severity of the injury.
Injured reserve could be an option, per Rapoport, which would wipe out at least eight weeks.
The right foot is the same one Olsen had surgically repaired last season, which cost the tight end eight weeks. After three straight 1,000-yard seasons, the 12-year pro has seen the last two years derailed by injuries.
The Panthers won Sunday's game, but Tuesday's news brings big concerns for the offense moving forward.