[BEEPbuilders] Defining Beep Interoperability Tests (Echo first?)

Bob Wyman bobwyman@firstrain.com
Thu, 29 Aug 2002 17:42:32 -0400

Gabe Wachob wrote:
> your proposal actually tests much more than 
> the Echo profile (at least steps 2 and 3).
	What I should have suggested was a series of scripts:
	1. The Basic Echo with a variety of packet sizes.
	2. A TLS tuning script which then runs the echo profile.
	3-x. A script for each SASL mechanism that then runs the echo
	These Echo tests should exercise the MSG/RPY message exchange
pattern fairly well. 

> we should also (in fairly short order) find another 
> profile (or invent a simple one) to get at least 
> minimal coverage of the message types.
	To test MSG/ANS/NUL, we could do a "Reverb" profile that would
be like Echo, except that it would chop the message payload into a
number of parts. For instance, the initiator might send 3K of data
broken into three parts and the listener would then send the same parts
back. As with Echo, the sizes of the messages would vary. Also, the
number of parts should be varied. A complete test suite would run this
test with TLS, SASL, and any other appropriate tuning options as well.
	A third profile would be defined to handle the MSG/ERR pattern.
This profile would, I think, be a bit more contrived than the first two.
Given the nature of the MSG/ERR pattern, it would consist of a number of
wired errors. (i.e. some bad XML, and some messages which direct the
remote implementation to generate a specified error.)

	Much would be left untested by such a simple set of tests. On
the other hand, much would be tested. I think these three profiles with
some appropriate scripts would be a good start to rapidly getting
effective interoperability testing.

		bob wyman