 |
|
d
a t a - c h a s e r
mission
updates |
|
|
|
TUESDAY
8/5 (two days to launch)
Student
teams have deployed to various control sites around the country:
-
DATA-CHASER
project manager Peter Illsley and team member Shankini Doraisingam
will be viewing the launch in person at Kennedy Space Center,
Florida (along with a half-dozen or so CSGC students who have
traveled to see the launch at their own expense). Immediately
following launch, Peter will travel to Goddard Space Flight
Center in Virginia, and Shankini will return to Colorado to
assist in payload operations at CU.
-
Goddard
Space Flight Center is where NASA shuttle-small-payload operations
are carried out. DATA-CHASER operations manager Ryan Shepperd
and team members Colette Wilklow and Chris Sturm will be working
at GSFC for the duration of the mission, serving as an interface
between NASA and DATA-CHASER operations.
-
DATA-CHASER
students remaining at CU for mission operations include systems
engineer Jason Willis, and team members Cindy Conrad, Tom
Height, Cortney Henderson, Tim Leisy, and Diana Mann.
-
To
demonstrate distributed operations, an additional DATA-CHASER
operations site will be set up at the Jet Propulsion Laboratory
in Pasadena California. CSGC alumni Kathryn Schimmels and
Clark Snowdall will be running things there.
DATA-CHASER
personnel at all stations are currently performing final software
and network checkouts before the mission begins.
WEDNESDAY
8/6 (one day to launch)
Minor software
problems were noted during final operations setup; these have
been fixed by the DATA-CHASER student engineering teams now in
place at the University of Colorado and
Goddard Space Flight Center, VA. Ground software is now operational
and the teams are eagerly awaiting launch tomorrow ("Christmas")
morning.
KSC
reports that there are currently no technical issues which may
delay the launch of STS-85.
Bad weather is a possibility; currently there is a 40% chance
that the launch may be affected. If the launch is delayed, the
next opportunity comes Friday morning 8/8.
THURSDAY
8/7 (launch, flight day 1)
5:41AM MDT:
"Well,
if anyone wanted to stop the shuttle launch, you missed your
chance. We (ASPC) just gave the T-3 hour 'go' and I didn't object
:) Now let's pray for the weather!"
-Ryan Shepperd
8:00AM MDT:
CSGC students
and staff gather together around the country to watch the launch
of STS-85.
Project
manager Peter Illsley and team member Shankini Doraisingam (along
with a half-dozen other CSGC students) are watching the launch
live at Kennedy Space
Center, Florida.
Payload
operations manager Ryan Shepperd and team members Colette Wilklow
and Chris Sturm are at the Goddard
Space Flight Center in Virginia.
A large
group of students and staff, including DATA-CHASER systems engineer
Jason Willis, and operations team members Cindy Conrad, Tom
Height, Cortney Henderson, Tim Leisy, and Diana Mann, are watching
from the University of Colorado.
And several
CSGC alumni, Clark Snowdall and Kathryn Schimmels (former DATA-CHASER
project manager) are watching from the Jet Propulsion Laboratory in
California.
8:41AM MDT:
STS-85
lifts off from Kennedy Space Center
in Florida. The launch goes extremely smoothly despite threatening
weather, and the DATA-CHASER team is all smiles. Now the hard
work begins!
9:00AM MDT:
With Discovery
safely in orbit, DATA-CHASER
operations commence. Students at the UPOCC (University Point
of Command and Control, University of Colorado) and the ASPC
(Shuttle Small Payload Operations at Goddard Space Flight Center,
VA) communicate via NASA voice loops and Internet network connections.
MET (mission
elapsed time) 0/3:15 (0 days, 3 hours, 15 minutes after launch)
The first
opportunity to contact DATA-CHASER after launch came during
the IEH-2 powerup sequence. CHASER was powered up at MET 0/3:09,
and DATA was powered up at MET 0/3:38. Both systems powered
up properly. Contact was established with DATA, and 10 "pings"
(a quick test of the end-to-end network connecting CU and DATA-CHASER)
were sent and properly returned. An engineering scan (a comprehensive
survey of the on-board sensors) was performed and everything
looked nominal. The team is now awaiting the door-opening sequence
at MET 0/10:00 and the first viewing opportunity (a look down
at Earth to test the instruments) at MET 0/16:00.
FRIDAY 8/8
(flight day 2)
The
DATA-CHASER team has been successfully communicating with
the orbiting payload as opportunities appear in the mission
timeline. Despite minor problems, the DATA segment of the
project has been an outstanding success, allowing students
at the University of Colorado to control orbiting hardware
via the Internet. The science instruments in CHASER are
sending back data which is currently being evaluated.
At MET
(Mission Elapsed Time) 0/10:00 (0 days, 10 hours), DATA-CHASER
had it's first Earth-viewing opportunity. The instruments
in CHASER were designed to look directly at the sun and
therefore cannot image the much dimmer Earth, but such opportunities
allow the computers, software, and networks to be tested.
At MET
1/3:15 (one day, 3 hours, 15 minutes) the first solar-viewing
opportunity of the mission occurred. A science-scan was
taken, and team scientists are reviewing the resulting data.
FARUS, the main science instrument in CHASER, is close to
20 years old and a veteran of many rocket and shuttle flights.
It may be showing it's age, as the resulting data did not
show the features that the project scientists were expecting.
However, the format of the data appears to be that of an
operating instrument, so project members are hopeful that
it is just being finicky and will perform better as the
mission progresses.
The
team will continue to take scans during science opportunities
as the mission progresses, and begin testing advanced automation
software. Minor problems with the ground software are also
being addressed by project software engineers Sam Siewert
and Joe Antell.
|

DATA-CHASER is just one of the experiments
on the IEH-2 bridge,
which is located at the rear of Discovery's cargo bay. Graphics
courtesy NASA.
|
|
The POCC (Point of Command and Control) at the University
of Colorado. Present are DATA-CHASER Systems Engineer Jason
Willis, Operations Team member Tom Height, and Science Team
member Tony Colaprete. Students will be working in shifts
around the clock for the duration of the 10-day mission.
|
 |
MONDAY 8/11
(flight day 5)
The
DATA (Distributed Automation Technology Advancement) portion
of the experiment is going extremely well. The ground software
is running solidly (occasional database hangs are being
looked into), and the team is planning on testing advanced
autonomy software and additional remote Internet operations
sites later this week.
The
main concern for the CHASER portion of the experiment is
the FARUS (FAR Ultraviolet Spectrometer) instrument. Mechanically,
the instrument is operating correctly (the instrument consists
of a motorized grating which directs light at various wavelengths
onto a photomultiplier tube). The instrument is measuring
solar spectra, but in a very coarse fashion (the data coming
out is either 0 or full-scale, without any values in between).
The engineering team is currently monitoring the situation,
trying to determine whether this is a temporary or permanent
problem, and working on alternate measurement strategies.
The
operations team has also been busy monitoring the flight
computer, which does not operate reliably at temperatures
below -10 degrees C. Such an incident occurred Friday night
(August 8), when the computer refused to respond to ground
commands. After a few tense hours the payload warmed up
enough for the computer to function, and there have not
been any problems since. The team is currently keeping a
close eye on the payload's temperature, and keeping it warm
by operating auxiliary heaters and equipment in the DATA
canister.
Quotes
of the day:
"I
get cold, payload gets cold, no one is happy"
-Colette Wilklow at GSFC
"Everything is just peachy when I'm on shift"
-Chris Sturm at GSFC
|
DATA-CHASER
Systems Engineer Jason Willis and Payload Operator Diana Mann
in the CU POCC (Payload Operations Control Center). The students
at CU are commanding and operating the payload in realtime,
and are in continuous voice communications with the CU team
at the NASA Small-Shuttle Payload Operations at Goddard Space
Flight Center.

DATA-CHASER
Operator Diana Mann in the pilot's seat. Through the Graphical
User Interface (GUI) which is part of the mission operations
software, commands can be sent to the orbiting hardware, and
the resulting data can be viewed and analyzed. In addition
to screen display, the data is stored in a database for further
study.
|
   
CHASER's
door opening (rear, center), as caught by the Shuttle video
system at the rear of the cargo bay. Video courtesy NASA.
|
|
|
|
WEDNESDAY 8/13
(flight day 7)
The
DATA-CHASER mission is going extremely well, despite continuing
difficulties with CHASER's FARUS instrument. It is returning science
data, and the science team can distinguish wavelengths in the data,
but not intensities at those wavelengths. Though this problem limits
the science benefit of the data, FARUS is still supporting the DATA
experiment, which is the primary objective of this project. The
team is continuing to evaluate the condition of the instrument,
and the flight software team is working on possible remote fixes.The
team has also been dealing with a cold flight computer. Once again
the temperature of the 68040 main processor dropped below 0°C causing
the computer to temporarily stop functioning. DATA-CHASER systems
engineer Jason Willis has dubbed this the "siliconsicle"
mode. The team has since been careful to turn on extra equipment
and heaters in the DATA canister to keep the processor warm. Project
scientist Tony Colaprete explains that the cargo bay is normally
kept between 0°C and 30°C, but has been particularly cold on this
mission due to the large amount of limb and deep-space pointing
required by the various experiments on board.The DATA-CHASER team
is gradually adding more autonomy to the mission operations system.
Where all early mission operations were performed manually (so that
problems could be quickly spotted and solved by human operators),
much of the repetitive data-gathering and evaluation operations
are now performed by software. SELMON (SELective MONitoring), a
software package developed by the Jet Propulsion Laboratory, is
being used to search incoming engineering data for subtle trends
and correlations which could signal trouble down the line. This
type of software will be extremely important for future deep-space
missions, where contact time is at a premium and spacecraft must
be able to take care of themselves. The Colorado
Space Grant Consortium is evaluating this and other software
as part of it's EEMOS (End to
End Mission Operation System) concept.
Workin' on the
night shift...
|

|

|
| Day
shift operator Diana Mann hands off operations to night shift
operators Cortney Henderson and Tim Leisy (hi mom!). Students
are working 12-hour shifts, 24 hours a day for the 10-day mission. |
Operators
Shankini Doraisingam and Cortney Henderson burning the midnight
oil. Since the shuttle orbits every 90 minutes, just as much
science can happen in the middle of the night as the middle
of the day. |
|

|

|
| Shankini
Doraisingam in the driver's seat. Shankini sends a special "hello"
to her parents on the other side of the world (she's sneaking
in a FARUS scan of Malaysia as we speak). |
Cortney
Henderson crunching the numbers. Cortney turned 21 during this
mission, but didn't have time to celebrate! (We'll have a big
party for her sometime after Discovery's landing on Sunday.) |
|

|

|
| Tim
Leisy in the NASA communications center down the hall. He has
direct access to the NASA voice loops, and relays commands and
queries between the DATA-CHASER teams at Goddard and in the
UPOCC. He also has his own operator's console so he can operate
and monitor the payload as well. |
Space
Grant is borrowing this room from Bioserve Space Technologies, another CU space experimentation center, who use
it for their frequent shuttle missions. Thanks Bioserve! |
Quotes
of the day:
"Dig
it!"
Operator Diana Mann's standard communications signoff
(a vast improvement over the NASA standard: "copy")
"Wish
I could give you good news to post about FARUS, but I'm afraid
it's cooperating as well as my 1982 Mercury Lynx does in subzero
weather."
Ryan Shepperd, who should know
Colette Wilklow
on climbing stairs:
"This is why I'm tired at the start of the shift"
Ryan Shepperd,
after waking up, in response to "How are you?"
"Judging by the amount of toothpaste on my fingers instead
of the toothbrush, not very well."
FARUS, after
waking up, in response to "farus_scan_A = 1"
"46 46 20 30"
Tim Leisy
on repetition:
"Roger, CHASER, Roger"
FARUS on repetition:
"20 30 30 20 30 30"
"The
system was crashing because of unwanted gui children that needed
to be killed. Jason killed them and Joe went to the movies"
Shankini's log for MET 05/11:50:00
FRIDAY 8/15
(flight day 9)
Flight day
9 has been a cold one for the DATA-CHASER team. Due to the large
amount of deep-space pointing done by Discovery (several of the
experiments on-board have scheduled viewing time for objects such
as Jupiter and the Hale-Bopp comet), DATA's 68040 main processor
has been below 0°C for much of the day. The processor shuts down
at such temperatures, keeping communications at a minimum. (When
Discovery points the cargo bay towards deep space, it is effectively
radiating towards infinity, and things get very cold very fast).
As CHASER's instruments were only designed to look at the sun,
this is not necessarily "down time" for science, but
the DATA portion of the project, which tests out advanced mission-operations
concepts, can always use engineering data from the spacecraft
to run through the ground software. More solar views are scheduled
for the near future, so DATA-CHASER should shortly be warming
up and running more reliably.
The DATA-CHASER
team has also been demonstrating the advanced mission-operations
system to viewers around the country. Several demonstrations were
made for the Jet Propulsion
Laboratory, who are EEMOS (End to
End Mission Operation System) development partners with the
Colorado Space Grant Consortium.
A remote-operations console was set up at JPL in Pasadena California,
and operators there (including CSGC alumni and former DATA-CHASER
Project Manager Kathryn Schimmels) were able to command the spacecraft
and look at the resulting data from that location. Despite minor
network problems (the present level of software does not have
all the network error-correction code that future versions will
have), the demonstrations went very well.
It's only software...
|

|

|
| Operator
Cindy Conrad analyzing data generated by SELMON (SELective MONitoring),
a software package developed by the Jet Propulsion Laboratory
which is being used to search incoming engineering data for
subtle trends and correlations which could signal trouble down
the line. Our primary data analysis tool is IDL
(Interactive Data Language). |
Operator
Tom Height works with Advisor Jack Faber on SCL (Spacecraft Command Language)
scripts which are used to test operations automation. These
scripts can be used to schedule operations events, check spacecraft
health and status, and respond to anomalies. |
|

|

|
| Jack
Faber works with Systems Engineer Jason Willis on the "Beacon
Mode" functionality of the operations system. Beacon Mode
is a high-level overview of spacecraft status being developed
by JPL for use on
deep-space missions. |
Beacon
mode gives the spacecraft an overall "green", "yellow",
"orange" or "red" status based on current
and predicted events. Although DATA-CHASER is not a deep-space
mission, this functionality was added to demonstrate the flexibility
of the End-to-End
Mission Operations System being developed by CSGC in conjunction
with JPL. |
|

|

|
| Jack
Faber, Jason Willis, and Operator Diana Mann set up a remote-operations
demonstration with JPL. Despite occasional network trouble (DATA-CHASER's
software doesn't have all the network error detection and correction
features that a commercial product would), the demonstrations
went very well. |
Jason
Willis works with the PLAN-IT II scheduling software developed
by JPL. This sophisticated software allows automated scheduling
of events (with conflict resolution if required), and acts as
a constraint database (for example, preventing communications
operations during loss-of-signal events). |
Quotes
of the day:
When Eileen
Jemison came in to work Wed. morning, she spotted Systems Engineer
Jason Willis looking a bit raggedy. She cried, "Oh
my gosh, what's that?!?! Beat it with a stick!!"
In a similar
vein, Trish Dunbar commented to Software Engineer Joe Antell:
"Gee, Joe, you're looking rode hard and put up wet!"
Jason upon
forgetting to save a file:
Jason: "I am an idiot... no comment."
Shankini: "Well, sometimes we are the best judge
of ourselves."
Jason on the
ops for this mission:
"First the distributed part, now the autonomy, the
3rd stage will be ops by mental telepathy."
SUNDAY 8/17
(flight day 11)
The last days
of operations for DATA-CHASER have gone extremely well. The payload
made its last solar observations (warming up from the below-zero
conditions of the last several days), and demonstrated full autonomy
in the ground software. The addition of "beacon" functionality
to the operations software has closed the loop; it can now autonomously
request information from the spacecraft, analyze the resulting
data, and give a high-level assessment of the spacecraft's health.
(The student operators have even commented on how little they
now have to do, as compared to the beginning of the mission where
everything was run manually). This was exactly the goal
of the DATA (Distributed Automation Technology Advancement) project,
and everybody involved is extremely pleased with the results.
As of this
writing, NASA has powered down the IEH-2 bridge containing DATA-CHASER
and is planning for a Monday-morning landing of Discovery at Kennedy
Space Center. For a while it looked like the mission might be
extended by a day, but preparations are continuing for deorbit.
Feelings are mixed; operator Shankini Doraisingam expressed regret
that the mission was over, but operator Diana Mann has had enough
12-hour shifts for the moment. All agree that this has been an
unforgettable three-year experience.
The
Away Team
|

|

|
| The
HHPOCC (HitchHiker
Payload Operations Control Center) at Goddard Space
Flight Center, MD. This is the interface between
NASA operations and the experimenters; through the equipment
in this room, experimenters get to directly communicate with
their experiments onboard the shuttle. |
Although
the DATA experiment is to allow payloads to be controlled remotely
via the internet, this mission required a dedicated team on-site
to maintain the internet link, run (and sometimes debug) complex
software, and act as liaisons between the DATA-CHASER team and
NASA. Peter Illsley, Ryan Shepperd, Chris Sturm and Colette
Wilklow were up to the challenge. |
|

|

|
| Ryan
and Colette at the DATA-CHASER experiment station in the HHPOCC.
Like the operations center at CU, this station must be manned
around the clock. This room is the first place that all the
shuttle experimenters see the results of their work, so it can
be a very exciting place. |
Colette
on the voice loops. There are many channels of NASA communication
running simultaneously, including those between CU and the team
at Goddard. Although the equipment allows selective monitoring,
important information can come at any time from several different
places Picking up one voice out of the noise can be challenging,
to say the least. |
|

|

|
| Ryan
keeping a close eye in the code. The software systems which
make DATA-CHASER possible are extremely complex, and keeping
it running is a full-time job. |
Chris
Sturm and Colette look on as Ryan performs some UNIX magic.
This team has performed flawlessly under extremely difficult
conditions, and our virtual hats go off to them. All photos
by Peter Illsley, who also happens to be DATA-CHASER's project
manager. |
Quotes
of the day:
"All
personnel, please check for a fire in your area."
- The OD at Goddard, in a calm voice. (That fire is still
missing:)
Ryan, on authority:
Chris: "Are you sure you know how to drive this thing
[ultra-sparc greco]?"
Ryan: "Can I drive? I have the keys to this
thing!"
(moments later, the software crashed)
"Ha
lo lo!"
- Colette's standard greeting
Yes,
I do scoff! In fact, I bite my thumb at thee!
-Ryan in a frustrated moment
"Sleep
is a state of mind... the mind resides in the head.... you need
a head to sleep, you can't sleep without a head"
-Shankini while on the late shift suffering from lack of sleep
"Shankini
is just dead wrong. A headless moth can sleep just as well as
a headfull moth. In fact, it sleeps better because there is no
danger of snoring"
-Ryan
"Sleep
is really not sleep without the capability to dream.
Or to wake up."
-Shankini
"In
order to know if the moth were dreaming and hence sleeping, you
would have to wake it up. And that would be rude---especially
if it were dead. Restricting sleep to such a narrow definition
as the ability to wake up is a rather biased Malasiancentric view
of life. If I can't wake up, does that make me dead? No, of course
not. In fact, I've carried on a perfect conversation with Gabe
about Floridian hotel checkout procedures while completely asleep.
I was not awake and, contrary to the grimaces of my co-workers,
I was NOT dead."
-Ryan
(At this point
communications became garbled; we assume that both Ryan and Shankini's
shifts ended, and they are now resting comfortably. -Ed)
DATA-CHASER
STATUS FOR MONDAY 8/18:
The DATA-CHASER
team got an early-morning surprise when space shuttle Discovery
got a landing day wave-off due to fog at the Kennedy Space Center
landing site. The landing has been rescheduled for Tuesday morning
at 6:08 AM CDT. In the meantime, the payload bay was reopened,
the IEH-2 bridge was powered on, and all the on-board experiments
get an extra day to operate. DATA-CHASER is making use of the
time by giving its full-up autonomy package a thorough and extended
workout. "We're generating megabytes of data" commented
systems engineer Jason Willis.
The extra
day only adds to an already successful flight. Among other things
DATA-CHASER has demonstrated a number of tools, technologies,
and approaches to determine if they would be useful for the End-to-End
Mission Operations of future space missions. These flight demonstrations
include:
- Fault
Management
This includes fault monitoring, detection, isolation
(i.e., search to determine the cause of the problem), reaction,
and reporting. This was accomplished by a command and control
tool, SCL, and a selective monitoring tool, "SELMON",
located on the ground.
- Operations
Flexibility
Lessons learned early in the mission should result
in updates to operations later in the mission. This demonstration
was accomplished with project updates of the ground system consisting
of new scripts, rules, constraints, sequences, and operations
procedures.
- Self-Commanding
Self-Commanding consisted of: health, safety, and resource
utilization monitors to react to anomalies and command errors;
constraint checks to prevent commands that are not correct or
not appropriate for specific situations; automatic command sequences
to react to health / safety / resource problems and anomalies;
and automatic command sequences to react to detected opportunities
and events. The Systems Control Language (SCL) was used as the
control executive with command and data monitoring capabilities.
- Opportunity
Management
This demonstration include event monitoring, detection,
recognition, isolation, reaction, and reporting. It was accomplished
on the ground and prevented commands that were not correct or
not appropriate for that situation. Examples of such opportunities
could include unpredicted Sun-in-view, or instrument integration
time updates to optimize signal-to-noise ratios.
- Interactive
Operations
This is commonly known as "Joy sticking"
or the direct, interactive control of operations and instruments
by principal scientists and engineers located at their Science
Workstations, Central Workstation, Engineering Workstations,
or GSE. These operations should prevent things users shouldn't
do and enable things that users wish to do. This demonstration
also allows direct, interactive control of experiment subsystems
by subsystem engineers; direct control via single commands by
a ground user; and indirect control via scripts initiated by
a ground user. Example operations include pointing offset corrections
and fine-tuning integration times for various operational modes.
- Operations
by Users at Distributed Sites
Users at GSFC, Colorado, and JPL are involved during
the mission. This included demonstrating the capability of the
Security approach to verify commands sent through the firewall
to the payload. A hierarchy of command contentions and interactions
also demonstrated during the mission, including the smooth hand-off
of the control hierarchy from one level to another. Finally,
the control of redundant data links and distribution of data
to multiple users have demonstrated.
- Migration
of Autonomy
This includes the complete migration of automation
of functions from ground to flight, i.e., from the ground testbed
to the ground control center. It also includes migration from
human-controlled operations to fully-autonomous operations.
This begins with strictly manual control, migrates to procedural
control with human concurrence, to fully-autonomous control;
and back to human-controlled (if desired).
- Use
of Integrated Planning and Scheduling
The planning and scheduling functions are integrated
as part of the ground system. It includes planning updates to
respond to new flight plans and schedules, planning updates
to respond to unexpected anomalies or events, generating new
schedules of events around these updates, and automatic generation
of SCL scripts. The DATA-CHASER Automated Planner/Scheduler
(DCAPS) is a JPL-developed planning and scheduling tool to receive
Shuttle timeline data, develop a constraint-free schedule, and
generate the SCL commands to control payload operations.
- Beacon
Functions
The Beacon Function relies on SCL and SELMON to assess
the performance of the overall payload. This assessment is relayed
to the ground operators at one of four overall states -- green,
yellow, orange, and red - to indicate the need to communicate
with the ground users.
- Student
Operations
DATA-CHASER was also a successful demonstration of
how students are capable of operating complex space payloads.
The
End-to-End Mission Operations System (EEMOS) used by DATA-CHASER.
Graphic by Sam Siewert, CSGC.
Quotes
of the day:
"Sometimes
wishes do come true........ EXTRA DAY !!!!!!!!!!!!!!!!!!!!!!!!!!"
- Shankini
"This
is all Shankini's fault!"
- Diana
Remark to
a not-so-excited-Ryan after hearing that the landing was no-go:
"Just think. Tomorrow we get to do this again and
you won't have that (expletive deleted) end of mission report
to write"
"Hey,
do you know anything about UNIX?"
- ISIR experimenter to Ryan :)
"We
have the Phantom S-band!"
- Pete upon discovering that we are getting data when
the FOC says we are in ZOE
"Why
are they trying to land so early in the morning? See, that's their
problem!"
- Chris on the landing delay
|
|
|
|
|