AmigaDos.SystemTagList()

This documentation will first of all focus on running a command using the shell and doing so in an asynchrone matter. At the same time it tries to catch the output of that executed command, using the pipefs handler in AROS.

Caveats: There is little documentation at all concerning this topic, let alone for AROS in specific. So, whenever there is some documentation, you have to read that literally and fill in the blanks yourself. Hopefully this documentation is able to fill in some of those blanks.

Introduction

As can be read in the AutoDocs, the SystemTagList() function provides a way to execute a command via the shell. But what is not so evident, is that there are some caveats as well as some undocumented features that are not very well described.

Parameters

Let's start with the parameters that can be passed to this function.

The tags

[Insert here explaination about the tags]

Piping

Asynchronous piping with AROS using pipefs: can only be done following strict rules, not following those rules simply breaks your code.

First of all in asynchronous mode, you cannot use an exclusive lock on the pipehandle. The ideal situation for reading and writing simultanously from and into the pipehandler would require that the writing end opens the handle in shared write mode, and the recieving reading end should use a shared read mode.

A close inspection at the documentation reveals that this leaves only one single possibilty when using normal DOSOpen functionality, namely opening the writing-end in MODE_READWRITE and the reading recieving end should open the handle in MODE_OLDFILE. At first one would think that the recieving reading end could also open the handle in MODE_READWRITE (which would give the revieving end write acces to the handle as well), but there is a little caveat revealed in the pipe: documentation:

102 Pipes behave in most respects like ordinary files.  Some differences follow:
103 Pipes block for writing (i.e., the write request is suspended) when the
104 pipe's buffer is full, and block for reading when the pipe's buffer is
105 empty.  Thus, pipes are sort of like bounded ram: files.  EOF is returned
106 for reading when the pipe's buffer is empty and no process has the pipe
107 open for writing.

Which, in case one would open the receiving end in MODE_READWRITE, would render the pipe-handle unusable as the last read-command done on the pipe-handle would let that read-command wait forever until no-one (in this case the receiving-end itself) has the handle open for writing. The only way that somewhat solves this, is using the NP_ExitCode tag and in that called code, close the pipe-handle of the receiving end (in order to let the receiver's last read command 'unlock' so it can continue. But by doing so, it would also make the receiver's routine useless as it has no acces to the pipe-handler anymore. Besides that, the last read done by the recieving-end with the last read on the handle would also contain garbled values towards the end of what the last read tells that was possible to read (as some functions return how many bytes/characters were read).

How to use SystemTagList() in practise

Executing shell commands using SystemTagList():

• #Synchrone
• #Synchrone and hidden
• #Synchrone, hidden and catching Output
• #Synchrone, hidden, catching Output as well as Errors
• #Executing Asynchrone and catching Output
• #Executing Asynchrone and catching Output as well as Error

Synchrone

[insert explanation here] [insert example here]

Synchrone and hidden

[insert explanation here] [insert example here]

Synchrone, hidden and catching Output

[insert explanation here] [insert example here]

Synchrone, hidden, catching Output as well as Errors

[insert explanation here] [insert example here]

Executing Asynchrone and catching Output.

[insert more information here]

It took me a while to figure out that when using pipefs, i read all about buffers and pipe being buffered. So i didn't figure at first, that using buffered reads/writes would mess up things. Only when i started to use unbuffered reads/writes i was getting somewhere and got things to work. Ofcourse milage may vary in/for different situations.

note: The parameterlist in NP_ExitCode routine is uncertain, so procedure CMDExitCode() could contain parameters based on what actually is being passed to it. Unfortunately i could only find one(!?) program using NP_ExitCode in AROS sourcetree, that used these exact parameters. But it could be possible that SegList Parameter is only passed when using this specific tag, just as NP_ExitData would perhaps add another parameter being passed to the procedure.

  Program RunCMDoo;
  
  {
    Name   : RunCMDoo V0.1
    Target : AROS ABIv0/i386
    Author : n/a
    Date   : 2013-09-15
    Goal   : Run a command using SystemTagList() and catch its output
    Usage  : RunCMDoo Command "[parameter1 parameter2 parameterN]"
  }
  
  
  {$MODE OBJFPC}{$H+}
  
  
  Uses
    exec, amigados, utility, tagsarray;
  
  
  Type
    BPTR              = LongInt;  // Quick fix to compensate for pointer
  
    TRCMode           =
    (
      rcm_output,     // Only use SYS_Output
      rcm_combined,   // Use Sys_Output and Sys_Error using the same handle.
                      // (impossible using SystemTagList() ?)
      rcm_both        // use SYS_Output and Sys_Error both using their own handle.
                      // (currently   bugs)
    );
  
  Var
    CommandHasEnded   : boolean = false;
    CommandExitCode   : longint = 0;
    CommandSegList    : BPTR    = 0;
  
  Var
    OutPipeRead       : BPTR;
    OutPipeWrite      : BPTR;
  
  Const
    OutPipeName       = 'PIPEFS:CmdOut';  // Name should be randomized or use * (* = untested)
  
  
  
  Procedure CMDExitCode(retcode: LongInt; SegList: BPTR); cdecl;
  begin
    Writeln('Enter - MyExitCode()');
  
    CommandHasEnded := true;
    CommandExitCode := retcode;
    CommandSegList  := SegList;
  
    Writeln('MyExitCode(): exitcode =', CommandExitCode);
    Writeln('MyExitCode(): seglist  =', CommandSegList);
  
    Writeln('Leave - MyExitCode()');
  end;
  
  
  
  Procedure RunCMDOutput(CommandToRun: String);
  var
    TagsList       : TTagsList = nil;
    Tags           : pTagItem;
    res            : LongInt;
    nread          : LongInt;
    OutPipeBuffer  : packed array[0..255] of char;
  begin
    OutPipeWrite  := DosOpen( OutPipeName , MODE_READWRITE);
  
    if (OutPipeWrite <> 0) then
    begin
      addtags(TagsList,
      [
        LONG(SYS_Input)       , nil,
  
        LONG(SYS_Output)      , OutPipeWrite,
        LONG(NP_CloseOutput)  , 1,
  
        LONG(SYS_Error)       , nil,
  
        LONG(SYS_Asynch)      , 1,
        LONG(SYS_BackGround)  , 1,
        LONG(NP_ExitCode)     , @CMDExitCode,
        TAG_DONE
      ]);
      Tags := GetTagPtr(TagsList);
  
      writeln('RUNCMD(): Executing SystemTagList()');
      res := SystemTagList(CommandToRun, Tags);
      If (res <> -1) then
      begin
        writeln('RUNCMD(): SystemTagList() returned value ', res);
        writeln('RUNCMD(): opening OutPipe for reading');
        OutPipeRead   := DosOpen( OutPipeName   , MODE_OLDFILE);
  
        if (OutPipeRead <> 0) then
        begin
          writeln('RUNCMD(): entering main loop for reading data from OutPipe');
          while true do
          begin
              writeln('RUNCMD(): start a buffer read from OutPipe');
              nread := DosRead(OutPipeRead, @OutPipeBuffer[0], 255);
              // -1 = error, 0 = EOF and >0 = number of bytes actually read.
              if (nread <> -1) then
              begin
                writeln('RUNCMD(): buffer read from OutPipe was succesfull');
                OutPipeBuffer[nread] := #0;
                writeln(pchar(OutPipeBuffer));
                if (nread < 255) then break;
              end
              else
              begin
                writeln('RUNCMD(): ERROR - buffer read failed, IoErr() = ', IoErr);
                break;
              end;
              // Safety check ?
              if CommandHasEnded then Break;
          end;
          writeln('RUNCMD(): exiting main loop that read data from OutPipe');
          // Close our Output Pipe reader.
          DOSClose(OutPipeRead);
        end
        else writeln('RUNCMD(): ERROR - Failed to open pipe for read acces');
      end
      else writeln('RUNCMD(): ERROR - Failed to execute command');
      // close outputwrite when error occurend when executing systemtags()
    end;
  end;
  
  
  
  Procedure RunCommand(CommandToRun: String; RCMode: TRCMode);
  begin
    writeln('enter - runcommmand');
  
    case RCMode of
      rcm_output   : RunCMDOutput(CommandToRun);
  //    rcm_combined : RunCMDCombined(CommandToRun);
  //    rcm_Both     : RunCMDBoth(CommandToRun);
    end; // case;
  
    writeln('leave - runcommmand');
  end;
  
  
  
  (*
      MAIN
  *)
  var
   i : Integer;
   S : String = '';
  
  begin
    writeln('enter');
  
    If paramcount > 0 then
    begin
      for i := 1 to paramcount
        do S := S + Paramstr(i) + ' ';
      Writeln('Trying to execute command "',S,'"');
  
      RunCommand(S, rcm_Output);
    end
    else     // Show usage/examples
    begin
      Writeln('RunCMDoo v0.1');
      Writeln;
      Writeln('usage:');
      Writeln('  RunCMDoo Command [Parameter1 Paramere2 ParamterN]');
      Writeln;
      Writeln('example:');
      Writeln('  RunCMDoo LD --help');
      Writeln('  RunCMDoo LD --wrong_parameter_on_purpose');
      Writeln('  RunCMDoo LD -v');
      Writeln('  RunCMDoo dir ram:#?');
      Writeln;
    end;
  
    writeln('leave');
  end.

Executing Asynchrone and catching Output as well as Error

[insert explanation here]

Current problems: Unable to determine which pipe needs to be read first, so in the end one would always end up in a deadlock because not knowing what comes first OutPut or Error. Choosing the wrong one would make you wait forever and can only be 'broken' by reading the other pipe (flushing did not help).

Unable to:

• use WaitForChar() as pipes are not in raw mode
• seems not possible to change the mode using SetMode() (returns error)
• unable to use fib for size of pipe as a pipe-file is always zero
• Seek() function does not seem to work on pipes, again unable to determine size
• unable to glue/merge OutPut and Error together as SystemTaglist() automatically (no override possible) closes the Output and Error handle. Since the handles are the same (when glued/merged) it would result in a memory freed twice error (amongst others).

[insert example here]