You are correct. My team includes only your direct reports. However, at the far bottom left corner of the team panel you will see a gear icon. Click on that you will will be able to choose to show "sub teams" (direct reports of direct reports) or not. Please refer to my screenshot.