|
|
| |
Mosel database and spreadsheet examples
All examples in this directory are explained in the whitepaper
'Using ODBC and other database interfaces with Mosel'.
Examples that have several versions obey the following naming convention:
'filename*.mos' where '*' =
[] - 'odbc' driver,
1 - working with text data files,
2 - ODBC connection using SQL statements,
3 - 'excel' driver (Windows only),
4 - Oracle (OCI) interface,
5 - 'xls'/'xlsx' driver,
6 - 'csv' driver
|
| Reading sparse data from text files, spreadsheets and databases, and from memory: Data input, initializations from, SQL, ODBC, spreadsheet, CSV
|
| |
|
| Type: |
Programming |
| Rating: |
3 (intermediate) |
| Description: |
Suppose we want to read in data of the form
from a file, setting up a dynamic array A(range,
range) with just the A(i,j)= value(i,j)
for the pairs (i,j) which exist in the file. Here
is an example which shows different ways of doing this. We read data
from
- differently formatted text files (example trio.mos described in the 'Mosel User Guide', Section 3.4 Reading sparse data): using 'initializations from', 'readln', or 'diskdata',
- spreadsheets or databases (examples duo.mos, duosheet.mos, duooci.mos,
duoexc.mos, and duodd.mos described in the Xpress Whitepaper 'Using ODBC and other database interfaces with Mosel') ,
- the application (C/Java) executing the Mosel model (examples datainputc.mos and datainputj.mos
into different arrays, and using writeln
show that the arrays hold identical data.
Accessing spreadsheets and databases
With initializations from data from spreadsheets or databases is read in a similar way as text files are accessed; switching between data sources can
be done by replacing the I/O driver name and the data source in the extended filename, such as "mmsheet.excel:data.xls" to access the spreadsheet data.xls or "mmodbc.odbc:data.mdb" to read from the MS Access database data.mdb and the
string "mmodbc.odbc:DSN=mysql;DB=data" could be used to access
a mysql database named data.
Instead of using an initializations block, we may read data from a database into a Mosel model by writing out the corresponding SQL statements (this format allows
fine-tuning of the queries and
gives access to a larger range of commands, in particular when working
with databases). When developing applications with ODBC/SQL statements, the user is advised to use the following two parameter
settings to obtain error messages and debugging information from ODBC:
setparam("SQLverbose", true)
setparam("SQLdebug", true)
The module mmoci (requires a separate license) defines a software-specific interface to Oracle databases that is used in a similar way as
the ODBC connection.
Using initializations from the extended filename will take a form like "mmoci.oci:myname/mypassword@dbname" and it is also possible to work with PL/SQL statements like the following:
OCIlogon('myname/mypassword@dbname')
OCIexecute("select Index_i,Index_j,Value from MyDataTable", A5)
OCIlogoff
Data input in memory using I/O drivers
Using initializations from it is also possible to read data into a Mosel model from the calling application (C/C++, C#, Java) using a combination of
the I/O drivers mem (data held in memory) and raw driver (data in binary format) in C, drivers dotnet and dotnetraw in C# programs, and drivers java and jraw in Java programs.
Other options for data input
Other possibilities of inputting data into a Mosel model include
- Static modules: data exchange between the calling application and a model via custom functions, using the Mosel Native Interface - see the
example of embedding the cutting stock problem in the Whitepaper 'Embedding Optimization Algorithms'.
- Custom format/custom data sources: the Mosel Native Interface allows the user to implement his own I/O drivers (to be used with
initializations blocks and all other Mosel functionality
involving file access, such as matrix export or model compilation).
Possible uses of such user-written drivers include
encoding/decoding of data on the fly or compression/decompression
of data.
See the Xpress Whitepaper 'Generalized file handling in Mosel' for
the complete code and documentation of an example implementing
a compression driver with the zip library.
Furthermore, the Mosel IO
driver pipe may be used to recover the output of an external
program. See the section on I/O drivers in the 'Mosel Libraries
Reference Manual' for more information.
|
| File(s): |
duo.mos, duosheet.mos, duoexc.mos, duodd.mos, duooci.mos |
| Data file(s): |
data.csv, data.xls, data.xlsx, data.mdb, data.sqlite, datadd.csv |
|
|
| Data output to text files, spreadsheets and databases, and to memory: Data output, initializations to, SQL, ODBC, spreadsheets, CSV
|
| |
|
| Type: |
Programming |
| Rating: |
3 (intermediate) |
| Description: |
This example shows different ways of writing out data from a Mosel model, namely to
- differently formatted text files (example trio_out.mos described in the 'Mosel User Guide', Section 10.2 File output): using 'initializations to', free-format output with 'write'/'writeln',
or 'diskdata' format,
- spreadsheets or databases (examples duo_out.mos, duosheet_out.mos, duooci_out.mos, duoexc_out.mos, and duodd_out.mos described in the Xpress Whitepaper 'Using ODBC and other database interfaces with Mosel'),
- the application (C/Java) executing the Mosel model (examples dataoutputc.mos and dataoutputj.mos
Accessing spreadsheets and databases
Using initializations to data is written to spreadsheets or databases through the ODBC connector in a similar way as text files are accessed; all
necessary SQL commands are generated automatically. With the extended filename to "mmodbc.odbc:data.mdb" the data
will be written to the MS Access database data.mdb and the
string "mmodbc.odbc:DSN=mysql;DB=data" could be used to access
a mysql database named data.
Instead of using an initializations block, we may write out data from a Mosel model directly with the corresponding SQL statements (this makes it possible, for
example, to clear the data table before writing to it - this functionality is not available with spreadsheets)
SQLconnect("data.sqlite")
SQLexecute("delete from MyOutTable")
SQLexecute(
"insert into MyOutTable (Index1,Index2,AValue) values (?,?,?)", A)
SQLdisconnect
When working with Excel spreadsheets it is recommended to use one of the dedicated spreadsheet drivers from the mmsheet module: with the excel driver, the spreadsheet may remain
open while writing to it from a Mosel model, in this case the data
written out to the spreadsheet does not get saved, making it possible
to re-run the application with different data sets without causing
any lasting modifications to your spreadsheet. All other spreadsheet drivers require the spreadsheet file to be closed.
The xls/xlsx drivers do not require an installation of Excel and are available for certain non-Windows platforms.
The module mmoci (requires a separate license) defines a software-specific interface to Oracle databases that is used in a similar way as
the ODBC connection provided through mmmodbc.
With initializations to an extended filename will take a form like "mmoci.oci:debug;myname/mypassword@dbname" and it is equally possible to use PL/SQL statements directly.
Data output in memory using I/O drivers
Using initializations to it is also possible to send data from a Mosel model in memory to the calling application (C/C++, C# or Java). This is done
using a combination of the I/O drivers mem (data held in memory) and raw driver (data in binary format) in C, drivers dotnet and dotnetraw in C# programs and drivers java and jraw in Java programs.
Other options for data output
Other possibilities of outputting data from a Mosel model include
- Use of the Mosel Libraries: information about the model objects is retrieved after the execution of the model is terminated. See the 'Mosel User Guide',
Part III Working with the Mosel Libraries and the example dataafterexec.c.
- Redirection of the output and error streams: to a file or to the calling application with the help of a callback function (I/O driver cb).
- Static modules: data exchange between the calling application and a model via custom functions, using the Mosel Native Interface. See the
example datainout.c.
- Custom formats/custom data sources: the Mosel Native Interface allows the user to implement his own I/O drivers (to be used with
initializations blocks and all other Mosel functionality
involving file access, such as matrix export or model compilation).
Possible uses of such user-written drivers include
encoding/decoding of data on the fly or compression/decompression
of data.
See the Xpress Whitepaper 'Generalized file handling in Mosel' for
the complete code and documentation of an example implementing
a compression driver with the zip library.
Furthermore, the Mosel I/O
driver pipe may be used to channel the output of a Mosel
model to an external
program. See the section on I/O drivers in the 'Mosel Libraries
Reference Manual' for more information.
|
| File(s): |
duo_out.mos, duosheet_out.mos, duoexc_out.mos, duodd_out.mos, duooci_out.mos |
| Data file(s): |
data.csv, data.xls, data.xlsx, data.mdb, data.sqlite |
|
|
| Dense vs. sparse data format: array types, noindex
|
| |
|
| Type: |
Programming |
| Rating: |
2 (easy-medium) |
| Description: |
The example 'indexeg' shows how to read data tables with different
formats from a spreadsheet:
- a dense 3x2 table
- a dense 2x3 table
- a sparse 2x3 table
A spreadsheet may be accessed from Mosel through an ODBC connection (using the odbc driver, indexeg.mos, or with SQL statements, indexeg2.mos), with the software-specific driver excel (indexeg3.mos), or with the portable mmsheet module that can be used to read and write xls, xlsx (indexeg5.mos) and csv (indexeg6.mos) files.
The first two methods (odbc driver and SQL statements) apply to spreadsheets and databases. In addition, for Oracle databases we show how to use the
software-specific connection provided by the module mmoci (indexeg4.mos).
|
| File(s): |
indexeg.mos, indexeg2.mos, indexeg3.mos, indexeg4.mos, indexeg5.mos, indexeg6.mos |
| Data file(s): |
indexeg.csv, indexeg.xls, indexeg.xlsx, indexeg.sqlite |
|
|
| Auto-indexation for single-dimensional arrays: auto-indexation
|
| |
|
| Type: |
Programming |
| Rating: |
2 (easy-medium) |
| Description: |
This example shows how to read single-dimensional data tables from a spreadsheet or database file that only contains the data
values. The data arrays must have an index set of type range that gets populated automatically based on the row count, either starting with the default value 1 or with a specified start
value. Several arrays can be populated via a single statement.
|
| File(s): |
autoindex.mos |
| Data file(s): |
adt.csv, adt.xls, adt.xlsx, adt.sqlite |
|
|
| Writing out solution values to text files, spreadsheets or databases: solution data output, 'evaluation of'
|
| |
|
| Type: |
Programming |
| Rating: |
2 (easy-medium) |
| Description: |
Solution values of decision variables or constraints need to be copied into an array. The contents of this array can then
be written to a spreadsheet or database using ODBC or software-specific drivers (or simply to text file using Mosel's default
output).
- Spreadsheets and databases can be accessed from Mosel through an ODBC connection using the odbc driver (soleg.mos) or with SQL statements (soleg2.mos).
- If Microsoft Excel is installed, the software-specific driver excel (soleg3.mos) can be used.
- For Oracle databases a software-specific connection is provided by the module mmoci (soleg4.mos).
- For generic spreadsheets it is recommended to use the portable mmsheet module that provides subroutines to read and write xls and xlsx (soleg5.mos) and csv (soleg6.mos) files.
As an alternative to the explicit copying of arrays of solution values you can use the keyword evaluation in initializations to blocks.
If preceded by the marker evaluation of solution values, results of Mosel
functions or other expressions can be used directly in the initializations to block as shown in the example initeval.mos.
|
| File(s): |
soleg.mos, soleg2.mos, soleg3.mos, soleg4.mos, soleg5.mos, soleg6.mos |
| Data file(s): |
soleg.csv, soleg.mdb, soleg.xls, soleg.xlsx, soleg.sqlite |
|
|
| Spreadsheets and databases: working with multiple data tables and arrays: data input/output, multiple tables, multiple data arrays
|
| |
|
| Type: |
Programming |
| Rating: |
2 (easy-medium) |
| Description: |
A database table (or a spreadsheet range) may contain in its different fields (columns) the data for several Mosel arrays
and inversely, a Mosel array may correspond to several data tables (ranges). We have here examples of various different cases:
- using the odbc driver,
- formulated with SQL statements,
- using the excel driver,
- using the oci driver,
- using the xls driver, and
- using the csv driver.
for three sets of examples:
- Reading several arrays from a single database table/spreadsheet range (multicol.mos)
- Outputting several arrays into a single database table/spreadsheet range (multiout.mos)
- Reading an array from several tables/ranges (multitab.mos)
|
| File(s): |
multicol.mos, multicol2.mos, multicol3.mos, multicol4.mos, multicol5.mos, multicol6.mos, multiout.mos, multiout2.mos, multiout3.mos, multiout4.mos, multiout5.mos, multiout6.mos, multitab.mos, multitab2.mos, multitab3.mos, multitab4.mos, multitab5.mos, multitab6.mos |
| Data file(s): |
multicol.dat, multicdd.dat, multicol.csv, multicol.mdb, multicol.xls, multicol.xlsx, multicol.sqlite, multiout.dat, multicol.csv, multicol.mdb, multicol.xls, multicol.xlsx, multicol.sqlite, multitab.csv, multitab.dat, multitab.mdb, multitab.xls, multitab.xlsx, multitab.sqlite |
|
|
| Formulation of SQL (selection) statements: Advanced SQL functionality, retrieve database and table structure
|
| |
|
| Type: |
Programming |
| Rating: |
3 (intermediate) |
| Description: |
If some advanced SQL functionality is required when accessing
a data source through an ODBC or OCI connection it will in general
be necessary to formulate the corresponding SQL statements
directly in the Mosel model instead of using the corresponding IO
driver in initializations blocks. We show here two examples:
- Selection statements using columns in
inverted order (ODBC: odbcinv.mos, Oracle: ociinv.mos)
- SQL with conditional selection statements and functions
(ODBC: odbcselfunc.mos, Oracle: ociselfunc.mos)
- Retrieving information about database table structure: list of database tables, list of field names, primary keys (odbcinspectdb.mos)
- Implementing 'stop on error' behaviour for SQL statements (odbcchkstatus.mos)
|
| File(s): |
odbcinv.mos, odbcselfunc.mos, odbcchkstatus.mos, ociinv.mos, ociselfunc.mos, odbcinspectdb.mos |
| Data file(s): |
multicol.mdb, multicol.sqlite, ocidata.dat, odbcsel.mdb, odbcsel.xls, odbcsel.sqlite |
|
|
| Reading and writing records and lists: record data structure, list data structure
|
| |
|
| Type: |
Programming |
| Rating: |
3 (intermediate) |
| Description: |
This set of examples shows how to work with advanced data structures when accessing data in spreadsheets and databases:
- reading in records (recordin*.mos)
- writing out records (recordout*.mos)
- reading and writing lists (listinout*.mos)
The basic model version uses an ODBC connection to spreadsheets or databases through the odbc driver, model version (2) uses ODBC through SQL statements, and model version (3) uses the software-specific driver excel to access Excel spreadsheets. Model version (1) shows how to work with Mosel's text data file format.
|
| File(s): |
recordin.mos, recordin1.mos, recordin2.mos, recordin3.mos, recordin4.mos, recordin5.mos, recordin6.mos, recordout.mos, recordout1.mos, recordout2.mos, recordout3.mos, recordout4.mos, recordout5.mos, recordout6.mos, listinout.mos, listinout1.mos, listinout2.mos, listinout3.mos, listinout4.mos, listinout5.mos, listinout6.mos |
| Data file(s): |
recorddata.dat, recorddata.csv, recorddata2.csv, recorddata.mdb, recorddata.xls, recorddata2.xls, recordout_templ.xls, recorddata.xlsx, recorddata2.xlsx, recorddata.sqlite, listdata.dat, listdata.csv, listdata.mdb, listdata.xls, listdata.xlsx, listdata.sqlite |
|
|
| Reading and writing external types: dates and time: Types date, time, datetime; data input/output
|
| |
|
| Type: |
Programming |
| Rating: |
3 (intermediate) |
| Description: |
The types 'date', 'time', and 'datetime' are defined by the Mosel module mmsystem. This module also defines several subroutines for handling these types (see model file dates.mos) and Mosel's text data file
format accepts these types. Special functionality for working with the date and time types is also implemented by the ODBC,
Excel, and Oracle (OCI) interfaces - their use is shown in this set of examples.
- Spreadsheets and databases can be accessed from Mosel through an ODBC connection using the odbc driver (datesinout.mos) or with SQL statements (datesinout2.mos).
- With Microsoft Excel spreadsheets we recommend to use the software-specific driver excel (datesinout3.mos).
- For Oracle databases a software-specific connection is provided by the module mmoci (datesinout4.mos).
- Excel spreadsheets can also be accessed from Mosel using a dedicated IO driver xsl from the mmsheet module. The driver supports manipulating excel files on various platforms without having to install Microsoft Excel (datesinout5.mos).
- Module mmsheet also provides an IO driver for csv files (datesinout6.mos).
|
| File(s): |
datesinout.mos, datesinout2.mos, datesinout3.mos, datesinout4.mos, datesinout5.mos, datesinout6.mos |
| Data file(s): |
datetime.csv, datetime.mdb, datetime.xls, datetime.xlsx, datetime.sqlite |
|
|
| Reading 3-dimensional arrays: Multidimensional array input, option partndx
|
| |
|
|
|
|