In this scenario I have 3 Excel workbooks open, but only one Taskbar button. The 1st Taskbar above is an example of all Excel windows grouped together (the default setting).
NOTE: I used Vista Ultimate and Excel 2007 in this tutorial. If you have any comments or questions please post them below. The instructions are the same for resetting this option to the default setting. If you are like me and you are looking for a way to ungroup the Excel windows into their own Taskbar buttons follow the instructions below. This can be tedious because you have to use the program menu or Ctrl +Tab to switch between workbooks. The default setting for this feature groups all Excel windows (or workbooks) into one Taskbar button.
I often work with multiple Excel windows at any given time and I have found that I work best with Excel when all open windows are displayed on the Windows Taskbar ( see screenshot below for illustration).