Up : MAXI Data Analysis Previous : Preparation Next : Tips, Tricks, and Traps

Data analysis

A quick instruction of FTOOLS-based standard analysis with MAXI archive data obtained from DARTS at ISAS. Users basically need to run only 2 scripts, mxdownload_wget for data download, and mxproduct for data analysis. For more detailed information about analysis, please see the Tips, Tricks, and Traps page.

The old page that we described data analysis before HEAsoft 6.25 release is here.


The MAXI team
Please send questions, comments, and bug reports to : maxihelp@ml.ac.jaxa.jp

1. Setting up

First of all, make sure to set up the essential environmental variables in your (UNIX-)Shell environment for HEAsoft and CALDB Preparation page.

2. Downloading the data

Here we are assuming you download a small part of the MAXI archive data. If you want a significant part of the entire archive, such as a half of it. Alternatively, you can choose to download the entire MAXI archive in your local disk, using the following command (or similar). The total data size is roughly 1.4TB as in February 2017.
wget -q -nH --cut-dirs=1 -r -l0 -c -N -np -R 'index*' -erobots=off --retr-symlinks https://darts.isas.jaxa.jp/pub/maxi/mxdata/obs/

Change directory to your chosen working directory for your analysis; suppose it is $HOME/analysis/ Then, run the script mxdownload_wget, specifying the coordinates (in J2000) of the center of your chosen region, radius (optional), period, and instruments/modes (optional). The MAXI archive data since 2009-08-15 (MJD=55058) is available.

The format of mxdownload_wget is,

mxdownload_wget -coordinates RA,DEC -date_from TSTART -date_to TSTOP (YYYY-MM-DD or MJD)  [and more OPTIONS]

The following is an example; it downloads the MAXI data for the area centered at the Crab Nebula for the period between 2010-01-01 and 2010-01-31.

 cd $HOME/analysis/
 mxdownload_wget --coordinates 83.633083,22.0145 -date_from 2010-01-01 -date_to 2010-01-31

Note that the example above would download the data up to 2010-02-01, a day after the specified date of 2010-01-31, because of the requirement by the pipeline software mxproduct, which uses a (tiny amount of) calibration data up to the day after the end-date of the data to process. All the MAXI data that satisfy the specified conditions are downloaded from the MAXI archive data-server (or the server you specify) to the sub-directories of the current directory.

MAXI does not monitor all the 768 sky regions everyday, some data of some regions for some days are not available. Also, There is a lack of data for different reasons: MAXI shutdown, downlink problems, processing errors etc. The detailed information about those problems will be appeared on the MAXI home page.

A user can specify a radius and an instrument mode with options, -radius (default: 8.0 deg) and -instruments (default: gsc_low), respectively. The following is an example of downloading the SSC data for the radius of 10 degrees.

 mxdownload_wget -coordinates 10.00,-20.000 -dates 2010-01-01,2010-01-31 -radius 10 -instruments ssc_med

You can view a concise on-line help by fhelp mxdownload_wget. The two options -dryrun and -planonly are for dry-running (simulated run) and can be useful to see how it works and how many data files will be (attempted to be) downloaded.

The following is the directory structure created by mxdownload_wget

and the following is the file/directory structure after the pipe-line processing by mxproduct explained in detail in the next section:

where CWD refers to the current working-directory, (gsc_low, gsc_med, ssc_med) is the instrument_mode (data rate), [MMMMM] is the Modified Julian Day (MJD) in 5 digits, [C] is the counter-ID of the GSC (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, a, b), [D] is the detector-ID of the SSC (“h” or “z”), [NNN] is the “Healpix”-ID, as explained below, [Object] is the prefix specified by a user (object option of mxproduct), and [Inst] is one of (g_low, g_med, s_med), corresponding to (gsc_low, gsc_med, ssc_med), respectively.

Healpix is the library to define the regions on the sky, dividing the entire sky into regions with a similar area-size to one another. The data in the MAXI archive make use of Healpix system with NSIDE=8, RING format, which divides the entire sky into 768 regions. The J2000 celestial coordinates (RA, DEC) that MAXI uses are related to the Healpix coordinates (θ, φ) for each pixel-ID with the following formula:

 (RA, DEC) = (φ, π/2 - θ) [radian] (J2000).

3. Analyzing the data

Now you have the raw MAXI data in your local disk, you are ready to make data reduction, using one of MAXI FTOOLS, mxproduct, to get the spectra and light-curve with your choice of filtering selections.

3.1. Data reduction by mxproduct


The following is the simplest example of the data reduction of the GSC-low data of Crab to make the spectra and light-curves. In this example, mxproduct is run at the root data-directory, which contains the directory obs/ .

 mxproduct  83.633083 22.014500 2010-01-01 2010-01-31 object=crab

This will create the GSC science products of spectra for the default region, response file for them, and light-curves in the default energy band, all for the region around the coordinates of (RA, DEC) [J2000] for the period between TSTART and TSTOP. The science product files are stored in the directory ./products/ (or, user-specified name by the option, outpath), which is created if it does not exist.

The ftool help is given by

 fhelp mxproduct

or read through this document for the complete reference.


The format of mxproduct is,


mxproduct requires 4 mandatory parameters; the center coordinates (RA and Dec) and the start and end time. RA and Dec should be given in degree (J2000). The format of TSTART and TSTOP is either YYYY-MM-DD or YYYY-MM-DDThh:mm:ss, where YY, MM, DD, hh, mm, ss are year, month, day, hour, minute and second, respectively. When only YYYY-MM-DD is specified, TSTART is identical to YYYY-MM-DDT00:00:00 and TSTOP is identical to YYYY-MM-DDT23:59:59.

In addition, mxproduct accepts several optional arguments, each of which is given in the format of option=PARAMETER.

  1. Instrument/Mode
  2. Spatial filter
  3. Energy-band filter.

Here is the detail of how to specify optional parameters.

The GSC has 2 data types: gsc_low and gsc_med. Users must choose either of them. Our recommendation is gsc_low, which is the default. In case to choose the medium data, set the option gscdl=med.
Spatial filter (Region files, recommended)
You can supply the source and the background region files for GSC and/or SSC. See the options srcregfile_gsc, bgdregfile_gsc, srcregfile_ssc, bgdregfile_ssc. mxproduct runs without region files, in which case the background region is automatically chosen before and after the scan of the source. However, it is reported that the estimated background level is underestimated when region files are not specified. Thus, we recommend specifying the source and background region files.
The region-file format is the ds9 format in plain text with the fk5 coordinate system in degrees. When saving a region file in ds9, be careful to select coordinates in degrees (sexagesimal may be selected by default in ds9). Currently, acceptable shape of the source region is a simple "circle", and the background region is "circle - circle". The background region has to be within a radius of 8.0 deg from the center coordinates, which can be changed by adding an option, radi_o. See the examples.

Example for the source region file:
 # Region file format: DS9 version 4.1
 global color=green dashlist=8 3 width=1 font="helvetica 10 normal roman" select=1 highlite=1 dash=0 fixed=0 edit=1 move=1 delete=1 include=1 source=1
Example for the background region file, which is a "ring" surrounding the source region:
 # Region file format: DS9 version 4.1
 global color=green dashlist=8 3 width=1 font="helvetica 10 normal roman" select=1 highlite=1 dash=0 fixed=0 edit=1 move=1 delete=1 include=1 source=1

Energy-band filter
In mxproduct, multi-band light-curves are generated and stored in multiple extensions in a single output FITS file.
The default energy bands are 2.0–5.95 and 6.0–11.95 keV for GSC, 0.7–1.997 and 2.000–6.997 keV for SSC.
If you want to change the energy bands, you can supply a plain text file that describes your desired (multiple) energy bands. See the options ebandfname_gsc,ebandfname_ssc.
Each line of the file consists of a pair (Minimum, Maximum) of the energy in keV, separated by a single space.
The following is an example for specifying three energy bands, 2–5.95, 6–11.95, and 12–20 keV.

 2.0 5.95
 6.0 11.95
 12.0 20.0  
Output files

mxproduct generates the GSC and/or SSC science products (spectra, spectral response file, and light-curves) in the directory products/ (or, user can specify the name by the command-line option, "outpath") , as well as some auxiliary files in the directory trend/, both in the current directory. The default energy bands are 2.0–5.95 and 6.0–11.95 keV for the GSC, 0.7–1.997 and 2.000–6.997 keV for the SSC. The following is a set of example product filenames for GSC_Low, where the command-line option of object=crab is given. For GSC_Med and SSC_Med, replace g_low with g_med and s_med, respectively.

Event file
Spectra and responses
Scancur file (observation condition for the target)
Command log
mxproduct.log (created at the path you ran mxproduct)

During the processing, many intermediate files (such as, GTI files, spectrum files for individual cameras) are created in the same products (or user-specified name) directory. They are automatically left after the processing in default, unless cleanup=yes is specified. The diagram in the previous subsection summarizes the directory and file structure.

The following figures show examples of WMAP image of Crab for source and background regions without (default settings) and with the region files. Without region files, the default source and background data are selected by elongation angles from the target in a scan direction and in a direction perpendicular to the scan direction. The generated images are neither exposure-corrected nor background-subtracted.

From left to right, a source WMAP image and a background WMAP image without region files, a source WMAP image and a background image with region files.
The generated light-curves are not yet corrected for the effective area. The absolute count rate, especially in the low energy band, is underestimated by ~10%. The light-curves may show some jumps, which might be artificial, not real by the target. The light-curves for Crab are shown below, about 10% less compared to the result by MAXI on-demand process especially below 4 keV. 
The light-curves of Crab from 2010-01-01 (MJD=55197) to 2010-01-31 (MJD=55227) by ondemand (left) and by mxproduct (right).
Prefix of the filenames of science products. Avoid any of the characters of [-+*/()] and a space for NAME (especially, watch out for the hyphen “-”), which would cause Memory Error while calculating spectrum files, using mathpha in FTOOLS. Default is target.
gscdl=(low or med)
Specify med to use the GSC med-data (Default: low).
skip_gsc=(0(Default) or 1)
If 0 (default), processes all the available GSC data of the relevant mode as specified in gscdl. Otherwise, skips all of them.
skip_ssc=(0 or 1(Default))
If 0, processes all the available SSC data. Otherwise (default), skips all of them.
Directory for the locally mounted data files, that is, the path where obs/ exists. Default is ".".
mountdata=(0(Default) or 1)
(Internal use) Uses the locally-mounted whole archive if 1.
Directory where output files will be put. Default is "products/".
Filename of the region file for the source with the GSC in the ds9/fk5 format. Only a simple circle shape is accepted.
Same as srcregfile_gsc, but for the GSC background region. Only a simple circle - circle shape is accepted.
Same as srcregfile_gsc, but for the SSC source region.
Same as srcregfile_ssc but for the SSC background region.
Filename of the energy band for GSC light curves. See the previous subsection Overview for detail of the format.
Same as the option ebandfname_gsc but for the SSC.
cleanup=(yes or no(Default))
mxproduct generates a number of intermediate files, including GTI files, event files, spectrum files in the directory ./products/ (or user-specified name) during processing. In default, all these files are deleted when mxproduct completes the task or is halted/interrupted (except for the case the process is killed by the KILL signal). They are not deleted if this option is specified to be 1.
  1. The simplest form for the source1 at (RA, DEC)=(350°, −10°).
     mxproduct object=src1 350 -10 2010-01-01 2010-01-31
  2. Processing the data for the radius of 10 degrees of Crab Nebura, where the MAXI data directory obs/ is mounted in /My/Data/Maxi/.
     mxproduct object=crab mountdata=1 datapath=/My/Data/Maxi radi_o=10 83.633083 22.014500 2010-01-01 2010-01-31
  3. Specifying region files for source and background.
     mxproduct object=crab srcregfile_gsc=src.reg bgdregfile_gsc=bgd.reg 83.633083 22.014500 2010-01-01 2010-01-31
  4. Specifying energy bands for GSC light-curves in a text file, gsc_eband.list, a directory name for output files, test1/, and start/stop time with hh:mm:ss.
     mxproduct object=crab ebandfname_gsc=gsc_eband.list outpath=test1 83.633083 22.014500 2010-01-01T08:00:00 2010-01-31T20:00:00
  5. Example of the SSC data processing.
     mxproduct object=crab skip_gsc=1 skip_ssc=0 83.633083 22.014500 2010-01-01 2010-01-31

3.2. Data analysis examples

The GSC has 2 data types: gsc_low and gsc_med. Users must choose either of them, and cannot mix them in the data reduction (gsc_low is the default in mxproduct). GSC consists of 12 counters, with the counter ID = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, a, b. The recommended energy ranges are 2-20 keV and 2-30 keV for gsc_low and gsc_med, respectively. The SSC has only 1 data type: ssc_med. SSC has 2 cameras, SSC-h and SSC-z. We recommend to use the 0.7-7 keV energy band for SSC.
Width of the spectral bin is 50 eV for GSC, and 3.65 eV for SSC.
3.2.1. Spectral analysis

The spectra and response files generated by mxproduct are in the standard FITS format. You can use your favorite software to analyze them. The following is the standard entry-level procedure with the XANADU/HEAsoft spectral-analysis package XSPEC to view the background-subtracted spectrum.


 cpd /xw
 data 1:1 "crab_g_low_src.pi"
 back 1   "crab_g_low_bgd.pi"
 resp 1   "crab_g_low.rsp"
 setplot energy
 ignore bad
 iplot ldata
 log x on 1
 log y on 1
 label t Crab
 lwidth 5
 rescale x 1 10

3.2.2. Time-series analysis

The generated FITS file of the light-curves (crab_g_low.lc in the example above) contains the same number of the FITS extensions as that of the energy bands specified in the input text file (gsc_eband.list in the example above) with the ebandfname_gsc (or ebandfname_ssc) option. Each FITS Extension is named LCDAT_PIBANDn for the (n+1)-th line of the energy band in the input gsc_eband.list. For example, the FITS Extension LCDAT_PIBAND0 contains the light-curve for the energy-band of the first line in gsc_eband.list. Each bin-width of a light-curve is 40~200 sec, which corresponds to the period of a single MAXI scan in which the source is in the field-of-view. The light-curve bins are separated by the intervals between the two successional scans, which are ~92 min (= ISS orbital period) in most cases but can be as short as ~15 min.

You can plot the light curve(s) with your favorite software, specifying the TIME as X-axis, and the RATE and RERR columns as Y-axis and its error, respectively. The following is the example with the general plotting tool LCURVE in HEAsoft:

 lcurve nser=1 cfile1="products/crab_g_low.lc" window="-" dtnb=INDEF nbint=INDEF outfile=" " plot=yes  plotdev="/xw"

Go to Tips, Tricks, and Traps for more details.