1

Closed

Race condition in CommandShellHost when shell is restarted

description

When the CommandShellHost (cmd.exe) is restarted (or when switching from CommandShellHost to PowerShellHost), a race condition develops around asynchronous I/O for stderr and stdout.
 
Steps to reproduce:
 
Scenario 1 (Restart CommandShellHost):
  1. Start the VsCmdShell Window using the cmd.exe shell.
  2. Right-click in the VsCmdShellWindow and choose "Restart Shell" from the context menu.
     
    Scenario 2 (Switch to a different IShellHost implementation, like PowerShellHost):
  3. Start the VsCmdShell Window using the cmd.exe shell, or switch to the cmd.exe.
  4. Right-click in the VsCmdShellWindow and choose "Options".
  5. In the Power Toys/VsCmdShell section, change the "Shell Selection" dropdown to a host other than "Windows Command Shell (cmd.exe)".
  6. Click OK on the Options Dialog.
  7. Dismiss the VSCmdShell message box instructing you to restart the shell for the change to take effect.
  8. Right-click in the VsCmdShell window and choose "Restart Shell"
     
    In either scenario, the CPU will race in the devenv.exe process. Setting breakpoints in CommandShellHost.ErrorBeginRead(), CommandShellHost.OutputBeginRead(), CommandShellHost.OnErrorInput() and CommandShellHost.OnOutputInput() will reveal an extremely tight call loop that is causing the race condition.
Closed Apr 14, 2007 at 10:57 PM by mattgollob

comments

mattgollob wrote Apr 14, 2007 at 9:13 AM

Scenarios 1 and 2 both have the same root cause: Ending the cmd.exe shell. They are both listed here simply to illustrate that there is more than one workflow in which a user might encounter this issue.

wrote Apr 14, 2007 at 9:31 AM

Resolved with changeset 21172.

mattgollob_cp wrote Apr 14, 2007 at 8:44 PM

mattgollob wrote Apr 14, 2007 at 8:44 PM

Needs to be assigned to a release

wrote Apr 14, 2007 at 8:44 PM

wrote Apr 14, 2007 at 10:18 PM

wrote Apr 14, 2007 at 10:57 PM

wrote May 6, 2007 at 12:59 AM

wrote Feb 13, 2013 at 10:14 PM

wrote May 16, 2013 at 2:27 AM