Streams

Summary:
  • Streams are a sequence of data that can be read from or written to a resource.
  • Streams are unidirectional.
Streams are a sequence of data that your applets can either read from or write to. They create a generic interface for exchanging data with resources. Once you have created a stream, you usually do not have to worry about what is on the other end of it.
Streams only flow in one direction. Your program reads data from an input stream and writes data to an output stream. If there is a resource you want to both read from and write to (for example, a network socket) you need to open both an input and an output stream to it.

The Stream Classes

Summary:
  • Streams are classified according to their features.
  • Read data from input streams, write data to output streams.
  • Binary streams handle raw bytes, text streams handle text.
  • Buffered streams use buffers to reduce overhead.
There are a large number of different stream classes. The differences between these classes are based on the features they include, and their purpose.
These differences are:
The stream classes are named according to these different features. For example, an instance of TranscodingSeekableTextInputStream is an input stream capable of reading text encoded using a multi-byte scheme, in which data can be read from any point in the stream.

Byte Streams

Summary:
  • Use byte streams to read and write binary files.
  • read-open-byte opens a file as raw binary data.
  • Use objects of type byte to store data.
  • read and read-one read binary data.
  • read-bytes-from opens and reads a file in a single step.
  • Writing binary files is similar to writing text files.
Byte streams let you read and write raw binary data. The Curl® Runtime Environment does not attempt to interpret the data in any way. You primarily use these streams when you need to manipulate binary data (an image file, for example) at a low level. Usually, you read from and write to one of these streams in set blocks of bytes.

Reading Binary Files

You can open a file to be read as bytes with the read-open-byte procedure. This procedure returns an instance of the ByteInputStream class.
Reading from a binary stream is similar to reading from a text stream, with one exception. You must read values into objects declared as byte. You cannot use objects declared as char because the Curl® language's characters are more than one byte, since they can represent Unicode characters.
Since binary files aren't broken into lines, as text files are, you need to use the InputStream-of.read or InputStream-of.read-one to read data from the stream. (Note that ByteInputStream inherits from InputStream-of.)
The following example demonstrates reading a binary file (a user-supplied image file) into an array of bytes. The read method is used, which usually results in reading the entire file into the array in one call.

Example: Reading a Binary File into an Array of bytes
|| Get an input stream from a Url, read data into an array of bytes
|| report the number of bytes read.

{let results:TextFlowBox = {TextFlowBox}}

{let image-file-types:{Array-of FileDialogFilter} =
    {new
        {Array-of FileDialogFilter},
        {FileDialogFilter
            "Image files",
            {new
                {Array-of FileDialogTypeFilter},
                {FileDialogTypeFilter "bmp"},
                {FileDialogTypeFilter "jpg"},
                {FileDialogTypeFilter "jpeg"},
                {FileDialogTypeFilter "gif"},
                {FileDialogTypeFilter "png"}
            }
        }
    }
}

{paragraph
    Click the button below to choose an image file from
    your system. If you do not have one handy, there is
    a {monospace .gif} file located at {value
    {url "../../default/images/url-accessor-diagram.gif"}.name}
}

{VBox
    {CommandButton
        label="Choose an Image File",
        {on Action do
            {results.clear}
            let file-url:#Url = {choose-file filters=image-file-types}

            {if-non-null file-url then
                let my-input:#ByteInputStream
                {try
                    || Open the file
                    set my-input = {read-open-byte file-url}

                    || Read from the file. This usually will return the
                    || entire content of the file, if you are reading
                    || from the local file system.
                    let (buffer:#{Array-of byte}, count:int) = {my-input.read}
                    {results.add
                        {text Got {value count} bytes of data from the file.}
                    }

                 catch err:IOException do
                    {results.add
                        {text Error occurred while reading the file:
                            {italic {value err.value}}
                        }
                    }

                 finally
                    {if-non-null my-input then
                        {my-input.close}
                    }
                }
            }
        }
    },
    results
}
You can use the shortcut procedure, read-bytes-from, to read a file as raw binary data into an array. See the API Reference entry for read-bytes-from to learn more.

Writing Binary Files

Writing bytes to a file is similar to writing to a text file. To write a byte to a file:
  1. Use write-open-byte to open a file for writing.
  2. Use the write or write-one methods to write data to the file.
  3. Use the flush method to ensure everything is written to the resource.
  4. Use the close method to close the file.
Note: Be sure to properly close the file to which you are writing. If you do not, you may lose data.

Stream Buffering

Summary:
  • By default, all streams are buffered.
  • Most file opening procedures let you specify buffer size.
  • Sometimes, you may want unbuffered byte streams.
In a buffered stream, data read from and written to a resource is stored in a temporary buffer in memory. Since the overhead of accessing a resource (such as waiting to access a disk or for a host on the network) is often large, buffering can make your program more efficient.
By default, all of the common file opening procedures (such as read-open and write-open-byte) return buffered streams. These methods all have an optional buffer-size parameter that lets you tweak the size of the buffer used on the stream. You may opt for a larger buffer size, depending on the amount of data and the frequency with which you are writing that data.
Sometimes, when you are reading or writing large amounts of data, buffering can actually be more costly than directly writing to the resource itself. You can eliminate the buffer on byte streams by setting the buffer size to zero.

Seekable Streams

Summary:
  • Seekable streams let you access any part of the stream.
  • Only certain streams can be seekable.
  • Use isa Seekable to see if stream is seekable.
  • Need to cast stream to a Seekable counterpart.
The default stream classes only allow you to read or write to the current position within the stream. For example, you can only read from the start of a regular input stream, and continue sequentially to its end.
A Seekable stream lets you read from or write to different positions within it. You can use them to extract data from files, or construct files that need data stored within them at specific positions.
Only those streams coming from or going to resources that can be accessed non-sequentially can be seekable. Files on the local file system are seekable since their entire content can be accessed at once. Streams from sockets and files read via HTTP and sockets can only be read in linearly from start to finish, so they cannot be seekable.
The way to determine if a stream is seekable or not is to test if it is a Seekable:
{if my-stream isa Seekable then
    || ... use the stream as a seekable
}
If the stream passes this test, then you can cast it to an appropriate class of seekable stream (such as SeekableTextInputStream). Search for Seekable in the API Reference Manual for a list of the available seekable classes.

Seekable Methods

Summary:
  • seek lets you move within the stream.
  • seek-style-supported? ensures you can move relative to a certain point.
All of the seekable streams inherit three methods from Seekable. The most important, Seekable.seek is what you use to move to different positions within the stream:
{Seekable.seek offset:int64, from:SeekFrom}:int64
The offset parameter is how far to move within the stream. The from parameter is a member of the SeekFrom enumeration, which indicates the point in the stream from which offset is relative. See the SeekFrom entry in the API Reference Manual for a list of the positions that seek supports.
Before attempting to seek to any point within the stream, you should first make sure the stream supports seeking from the position you want. Some streams may not support seeking from all of the positions listed in SeekFrom. The Seekable.seek-style-supported? method lets you ensure that the stream can seek from the position you want:
{Seekable.seek-style-supported? style:SeekStyle}:bool
This method takes a member of the SeekStyle enumeration as a parameter. This enumeration is different from SeekFrom's, since there are several more combinations that need to be taken into account. See the entry for SeekStyle in the API Reference Manual for a list of the values for style.

Seekable Example

The following example demonstrates how you can determine if a stream is seekable, and if it is, use the seekable methods to move within it.
Note: This example uses character-encoding = "shift-jis" for the read operation. If the file you select is not compatible with this character encoding, the example throws an error.

Example: Using a Seekable Stream
|| Demonstrate using a seekable stream.
{let results:VBox = {VBox}}

{let text-file-types:{Array-of FileDialogFilter} =
    {new
        {Array-of FileDialogFilter},
        {FileDialogFilter
            "Text files",
            {new
                {Array-of FileDialogTypeFilter},
                {FileDialogTypeFilter "txt"},
                {FileDialogTypeFilter "curl"}
            }}}}

{CommandButton
    label="Select a Text File",
    {on Action do
        let file-url:#Url = {choose-file filters=text-file-types}
        {results.clear}
        {if-non-null file-url then
            let my-input:#TextInputStream
            {try
                || Open the file
                let my-input = {read-open 
                                   character-encoding = "shift-jis",
                                   file-url}
                let count:int = 0
                || See if the file is seekable.
                {if my-input isa Seekable then

                    || Get a reference to the file that is an appropriate seekable
                    ||stream class.
                    let seek-input:SeekableTextInputStream =
                        (my-input asa SeekableTextInputStream)

                    || Use the tell method to determine where we are in the stream
                    {results.add
                        {text Current position is {seek-input.tell}}
                    }

                    || Ensure we can seek from the start of the stream.
                    {if {seek-input.seek-style-supported? "start"} then
                        || Jump 20 characters into the stream.
                        {seek-input.seek 20, "start"}

                        || Find out where we are now
                        {results.add
                            {text Now at {seek-input.tell}}
                        }

                        || Read next 10 characters
                        let buffer:StringBuf =
                            {seek-input.read-one-string n=10}
                        {results.add
                            {text The next 10 characters in the
                                stream are: {quote {pre {value buffer}}}
                            }
                        }
                        {results.add
                            {text After reading, we're at
                                {seek-input.tell}}
                        }
                    }

                    || See if it is safe to seek from the end of the stream
                    {if {seek-input.seek-style-supported? "end"} then
                        || move 10 characters from the end of the stream. Note
                        || that seeking from the end requires a negative offset.
                        {seek-input.seek -10, "end"}

                        || Find out where we are now
                        {results.add
                            {text After moving to the last 10 characters,
                                we're at {seek-input.tell}}
                        }
                        || Read next 10 characters
                        let buffer:StringBuf =
                            {seek-input.read-one-string n=10}
                        {results.add
                            {text The last 10 characters in the stream are:
                                {quote {pre {value buffer}}} (note that not all of
                                the characters read are printable).}
                        }
                    }

                    || Again, make sure it's OK to move relative to
                    || the start of the stream
                    {if {seek-input.seek-style-supported? "start"} then
                        || Move back to the start
                        {seek-input.seek 0, "start"}
                        || read the whole buffer into a StringBuf
                        let (buffer:StringBuf, numchars:int) =
                            {seek-input.read-one-string}

                        {results.add
                            {text The entire stream is:
                                {quote {pre {value buffer}}}
                                and is {value numchars} characters long.}}
                    }
                }

             catch err:IOException do
                {results.add
                    {text An error occurred while accessing the file:
                        {italic {value err.value}}
                    }
                }

             finally
                {if-non-null my-input then
                    {my-input.close}
                }
            }
        }
    }
}

{value results}

Compressed Streams

Summary:
  • Use the DeflateByteOutputStream class to compress binary data for output.
  • Text can be compressed by using a transcoding stream to convert it to bytes which the compressed stream handles.
  • Use the InflateByteInputStream class to read in a stream of compressed data.
The Curl Runtime Environment supports reading from and writing to compressed data streams. You can use this feature to write smaller files to disk, or to compress data for more economical transmission over a slow network link.
The compression algorithm used by the compressed streams is DEFLATE Compressed Data Format. See RFC 1951 DEFLATE Compressed Data Format Specification version 1.3 for details.
There are two compressed stream classes: one for input streams (InflateByteInputStream) and one for output streams (DeflateByteOutputStream). These classes take an instance of a byte stream as a parameter, which is what the compressed stream will read from or write to.
These classes are part of the CURL.IO.ZSTREAM package, which is not one of the runtime's core packages. Therefore, your applets need to import this package before they can read and write compressed streams. For an explanation of importing packages, see import Expression.
To write data out to a compressed file:
  1. Open a ByteOutputStream to the file which will store the compressed data.
  2. Create an instance of the DeflateByteOutputStream class, passing it the ByteOutputStream you just created.
  3. Write data to be compressed to the DeflateByteOutputStream. It will compress the data, then send it to the ByteOutputStream which will write it to the file.
  4. When finished writing data, call DeflateByteOutputStream.close to close both the compressed stream and the underlying byte stream.
To compress text, wrap the DeflateByteOutputStream stream in a TranscodingTextOutputStream. The transcoding stream will turn the text into bytes which can then be sent to the DeflateByteOutputStream.
Reading a compressed stream is the reverse of writing one: create a ByteInputStream that reads from the file or other resource that has compressed data. Then create a InflateByteInputStream that reads from the input stream. When reading compressed text, create a TranscodingTextInputStream to read from the InflateByteInputStream and convert the binary data to text.
The following example will write the compressed contents of the TextArea to a file that you select, then read the compressed contents back in again, once as a regular stream of bytes to demonstrate that the data was compressed, and then using a compressed stream to demonstrate reading compressed text data.

Example: Reading and Writing Compressed Streams
|| Import the package containing the compressed streams
{import * from CURL.IO.ZSTREAM}


{let output-area:VBox = {VBox}}

{let my-text:TextArea = {TextArea
                            value="This is some text that the " &
                            "compressed stream will write. " &
                            "It repeats in order to make the text " &
                            "more compressible. " &
                            "This is some text that the " &
                            "compressed stream will write. " &
                            "It repeats in order to make the text " &
                            "more compressible. "
                        }
}

{VBox
    width=5in,
    my-text,
    output-area,
    {CommandButton
        label="Choose & Write File",
        {on Action do
            {output-area.clear}
            || ask user for a file
            let target:#Url = {choose-file
                                  style=FileDialogStyle.save-as,
                                  title="Demonstrate Compressed Streams"
                              }

            {if-non-null target then
                let output-stream:#TranscodingTextOutputStream

                {try
                    || Open the file for writing, connecting the output
                    || stream to a compressed stream, then connecting the
                    || compressed stream to a stream that will turn
                    || ASCII text into bytes.
                    set output-stream = {TranscodingTextOutputStream
                                            {DeflateByteOutputStream
                                                {write-open-byte target}
                                            },
                                            CharEncoding.ascii,
                                            true
                                        }

                    || Write the string.
                    let output-size:int =
                        {output-stream.write-one-string my-text.value}

                    {output-area.add
                        {text Wrote the string:
                            {italic {value my-text.value}}
                            which was {value output-size} characters
                            long.
                        }
                    }
                 catch e:IOException do
                    {output-area.add
                        {text color="red", An error occurred while
                            accessing the file: {italic {value e.value}}
                        }
                    }
                 finally
                    || Close the topmost stream, which will flush &
                    || close all of the underlying streams as well.
                    {if-non-null output-stream then
                        {output-stream.close}
                    }
                }
                {try
                    || Read file back in to see how large it is
                    let (bytes:{Array-of byte}, read-size:int) =
                        {read-bytes-from (target asa Url)}

                    {output-area.add
                        {text The compressed file size is:
                            {value read-size}
                        }
                    }

                 catch e:IOException do
                    {output-area.add
                        {text color="red", An error occurred while
                            accessing the file: {italic {value e.value}}
                        }
                    }
                }
                || Now let's read the file back in and decompress it.
                let input-stream:#TranscodingTextInputStream
                {try
                    set input-stream = {TranscodingTextInputStream
                                           {InflateByteInputStream
                                               {read-open-byte target}
                                           },
                                           character-encoding=CharEncoding.ascii
                                       }

                    let (input-buf:{Array-of char}, in-count:int) = {input-stream.read}
                    {output-area.add
                        {text
                            Read {value in-count} characters back from the
                            compressed file.
                        }
                    }
                 catch err:IOException do
                    {output-area.add
                        {text color="red", Problem reading compressed file:
                            {value err.value}
                        }
                    }
                 finally
                    {if-non-null input-stream then
                        {input-stream.close}
                    }
                }
            }
        }
    }
}

Serialization

The Curl Serialization API is defined in the CURL.IO.SERIALIZE package. It enables you to write values to a SerializeOutputStream, which converts them to a stream of bytes that represents the values, and read values from a SerializeInputStream, which converts the stream of bytes into the values it represents. You can read and write most Curl datatypes, and arbitrary Curl data structures, including references (pointers.)
Version 6.0 of the Curl API provides a number of enhancements to the serialization interface. The following list summarizes these changes.
The following example serializes 11 integers and writes them to a file, then reads and deserializes them using the deserialize primitive.

Example: Serialize Integers
{import * from CURL.IO.SERIALIZE}
{let stored-values:HBox = {HBox spacing = 5pt}}
{let read-ints:int}
{define-proc {save-and-restore target:Url}:void
    {with-open-streams
        out = {SerializeOutputStream {write-open-byte target}}
     do
        {for i:int=0 to 10 do
            {out.write-one i}
        }
    }
    {stored-values.clear}
    {with-open-streams
        in = {SerializeInputStream {read-open-byte target}}
     do
        {stored-values.add {TextFlowBox width = 3cm, "stored integers:"}}
        {for i:int=0 to 10 do
            set read-ints = {deserialize in, int}    
            {stored-values.add read-ints}
        }
    }
}
{VBox
    {CommandButton
        label="Select a File",
        {on Action do
            let target:#Url = {choose-file
                                  style = FileDialogStyle.save-as,
                                  title = "Demonstrate Serialization"
                              }
            {if-non-null target then {save-and-restore target}}
        }
    },
    stored-values
}
The next example serializes and deserializes an array of 10 integers. Note that the entire array is treated as a single data object.

Example: Serialize an Array
{import * from CURL.IO.SERIALIZE}
{let stored-values:HBox = {HBox spacing = 5pt}}
{let arr:{Array-of int} = {{Array-of int} 0, 1, 2, 3, 4, 5, 6, 7, 8, 9}}
{let read-arr:{Array-of int} = {new {Array-of int}}}
{define-proc {save-and-restore target:Url}:void
    {with-open-streams
        out = {SerializeOutputStream {write-open-byte target}}
     do
        {out.write-one arr}
    }
    {stored-values.clear}
    {with-open-streams
        in = {SerializeInputStream {read-open-byte target}}
     do
        set read-arr = {deserialize in, {Array-of int}}
        {stored-values.add {TextFlowBox width = 3cm, "stored array:"}}
        {for x:int in read-arr do
            {stored-values.add x}
        }
    }

}

{VBox
    {CommandButton
        label="Select a File",
        {on Action do
            let target:#Url = {choose-file
                                  style = FileDialogStyle.save-as,
                                  title = "Demonstrate Serialization"
                              }
            {if-non-null target then {save-and-restore target}}
        }
    },
    stored-values
}

Defining a Serializable Class

You can save and restore complex Curl data by defining your own serializable classes. Use the serializable modifier to indicate a serializable class. A serializable class writes and restores all serializable superclasses and all fields that are not identified as transient with the transient modifier.
There are some restrictions on defining serializable classes:



For more on object-serialize and object-deserialize see below and documentation for CURL.IO.SERIALIZE package.
The following code sample illustrates a simple serializable class that contains a string (name) and an integer (score). It simulates the use case of a game that stores the name and score of the highest scoring player. If a file containing scores exists, the applet restores an instance of the serializable class HighScore and compares the restored score to the new high score. If the new score is higher, it writes the class to the file. To create a working example, copy this code into an applet, and replace [...] with a valid URL.
{curl 8.0 applet}

{import * from CURL.IO.SERIALIZE}

|| A class representing a "high score".
{define-class public serializable HighScore

  field constant name:String
  field constant score:int

  {constructor {default name:String, score:int}
    set self.name = name
    set self.score = score
  }
}
|| NOTE: Replace "[...]" below with a valid URL.
{let high-score-url:Url = [...]}

|| Start with "no" high score.
{let high-score:HighScore = {HighScore "nobody", 0}}

{try
    || Read any old high score.
    {with-open-streams
        in = {SerializeInputStream {read-open-byte high-score-url}}
     do
        set high-score = {deserialize in, HighScore}
    }
 catch e:MissingFileException do
    || The file may not exist yet.
}

|| NOTE: An actual game should be played here, but for this example,
|| we will just assume that a player named "somebody" beat the previous
|| high score by ten points.
{let name:String = "somebody"}
{let score:int = high-score.score + 10}

{if score > high-score.score then
    || Remember the new high score.
    set high-score = {HighScore name, score}
    || Save the new high score.
    {with-open-streams
        out = {SerializeOutputStream {write-open-byte high-score-url}}
     do
        {out.write-one high-score}
    }
}
The following code sample saves and restores an array of the HighScore objects defined in the previous sample. Again, note that the entire array of HighScore objects is handled as a single data object. To create a working example, copy this code into an applet, and replace [...] with a valid URL.
{curl 8.0 applet}

{import * from CURL.IO.SERIALIZE}

|| A class representing a "high score".
{define-class public serializable HighScore

  field constant name:String
  field constant score:int

  {constructor {default name:String, score:int}
    set self.name = name
    set self.score = score
  }
}

|| A class representing a "high score list".
{define-class public serializable HighScoreList

  field constant high-scores:{Array-of HighScore}

  || Add a new high score, and keep the top ten, sorted.
  {method {add high-score:HighScore}:void
    {self.high-scores.append high-score}
    {self.high-scores.sort
        comparison-proc =
            {proc {hs1:HighScore, hs2:HighScore}:bool
                {return hs1.score >= hs2.score}
            }
    }
    {if self.high-scores.size > 10 then
        {self.high-scores.remove 10, length = self.high-scores.size - 10}
    }
  }

  {constructor {default}
    set self.high-scores = {new {Array-of HighScore}}
  }
}
|| NOTE: Replace "[...]" below with a valid URL.
{let high-score-list-url:Url = [...]}

|| The high score list.
{let high-score-list:HighScoreList = {HighScoreList}}

{try
    || Read the high score list, if it exists.
    {with-open-streams
        in = {SerializeInputStream {read-open-byte high-score-list-url}}
     do
        set high-score-list = {deserialize in, HighScoreList}
    }
 catch e:MissingFileException do
    || The file may not exist yet.
}

|| NOTE: An actual game should be played here, but for this example, we
|| will just assume that a player named "somebody" got a random score.
|| random number of points.
{let random:Random = {Random}}
{let score:int = {random.next-in-range 0, 999999}}

|| Add the new high score.
{high-score-list.add {HighScore "somebody", score}}

|| Save the high score list.
{with-open-streams
    out = {SerializeOutputStream {write-open-byte high-score-list-url}}
 do
    {out.write-one high-score-list}
}

Serializable Class Versions

If you change the data stored by a serializable class in a new release of a Curl application, you need to use define-serialization to provide an integer greater than zero as the class version number in the revised class definition. The object-serialize method writes the class version as part of the serialized data stream. The dafault class version number is zero.
The revised class also must provide an object-deserialize constructor which checks the class version of the incoming data and behaves appropriately. The following example adds a field called when to store the date and time of the high score.
{define-class public serializable HighScore

  field constant name:String
  field constant score:int
  field constant when:DateTime

  {define-serialization class-version = 1}

  {constructor public {object-deserialize in:SerializeInputStream}
    let cv:int = {in.read-class-version}
    set self.name = {deserialize in, String}
    set self.score = {deserialize in, int}
    {switch cv
     case 0 do
        set self.when = {DateTime}
     case 1 do
        set self.when = {deserialize in, DateTime}
     else
        {error "Unknown HighScore class-version ", cv, "."}
    }
  }

  {constructor {default name:String, score:int}
    set self.name = name
    set self.score = score
    set self.when = {DateTime}
  }
}
This line of code sets the class version to 1:
{define-serialization class-version = 1}
The object-deserialize constructor first checks the class version of the incoming data:
let cv:int = {in.read-class-version}
Then it sets the name and score fields with the incoming data:
set self.name = {deserialize in, String}
set self.score = {deserialize in, int}
Then it examines the class version, and sets the when field to the incoming data if the data was stored by class version 1. If the data was stored by class version 0, there is no incoming date information, so the applet uses the current date and time.
{switch cv
 case 0 do
    set self.when = {DateTime}
 case 1 do
    set self.when = {deserialize in, DateTime}
 else
    {error "Unknown HighScore class-version ", cv, "."}
}

Serializable Types

Objects of he following data types are serializable: