|
May 26
|
|
| Todd Boroson |
NOAO |
Finding the Goodies in the Sloan Survey Archive of QSO Spectra |
|
Apr 28
|
|
| Irene Barg |
NOAO SDM |
Lessons Learned from PostgreSQL Administration Course |
|
I attended a 5 day PostgreSQL Administration Course April 6-10, 2009 at the
Open Technology Group http://www.otg-nc.com in Morrisville NC. Our
instructor Chander Ganesan took us through wide range of PostgreSQL
administration tasks, from manual installation and configuration to
performance tuning, connection pooling, full text searching, replication
and more. This talk will be about the changes I made to our PostgreSQL
installations because of the many techniques I learned from this course.
|
|
Apr 21
|
|
| Chuck Gessner |
NOAO |
Risk Management for Programmers, Ergonomics for Everyone |
|
Apr 7
|
|
| Dick Shaw |
NOAO |
Doing Science with Virtual Observatory Tools, or What I did at the NVO Summer School |
|
The much trumpeted utility of the Virtual Observatory was put to a severe
test this past fall when a well-intentioned, diverse group of scientists,
programmers, ameture astronomers, librarians, and Public Outreach
professionals of all ages, shapes, and technical prowess embarked on
"hands-on" science projects at the NVO Summer School in Santa Fe,
NM. Our particular project sought to assimilate the results of multiple
public and private surveys to search for infra-red counterparts to variable
objects (including planetary nebulae) in the Large Magellanic Cloud.
Although we had some false-starts along the way, in the end we learned that
there are generally multiple ways to approach a problem with VO tools, and
that our success would have been significantly more difficult to achieve
without them. I will conclude with a summary our new scientific results.
|
|
Mar 17
|
|
| None (Conflicts with WildStars II and SARSEF) |
|
Mar 10
|
|
| Brian Harker-Lundberg |
NSO |
GPU-Accelerated Stokes Inversion for Solar Vector Magnetography using NVIDIA's CUDA Platform |
|
Modern GPUs are typically an order of magnitude more powerful than their
modern CPU counterparts. Combine this with the fact that modern GPUs are
able to execute many instructions independently and in parallel (since they
were designed for high-speed, low-latency graphics rendering), and what
results is hardware for scientific computation that has an inherently
fine-grained parallelism. This type of platform can offer tremendous
speedup in an application, provided the application can be efficiently
parallelized to run over multiple threads. This short talk outlines the
hardware and CUDA software platform we will be using to implement a
high-speed, thread-based method of inferring vector magnetic field
information from solar Stokes polarization profiles.
|
|
Feb 24
|
|
| Mike Fitzpatrick |
NOAO Science Data Management |
NVO Tools for Data Discovery and Access |
|
The NVO Data Discovery Portal is a suite of integrated web applications
meant to allow astronomers to find the data, services and tools available
within the VO that enhance their science. This talk will introduce the
newly released Portal elements and provide real-science examples of its
use.
|
|
Feb 3
|
|
| Andrey Yeatts |
WIYN |
The ODI Instrument Pipeline |
|
The ODI pipeline is modeled as an event-dispatch framework. We look at its
design and discuss the remote pipeline service and how it integrates with
the StarGrasp controller service.
|
|
Jan 27
|
|
| Bradford Castalia |
HiRISE Operations Center |
Maestro: Managing Conductor Networks of Automated Processing Pipelines |
|
This presentation will provide an overview of the Maestro software for
managing networks of Conductor pipelines with an emphasis on the software
technology issues involved along with examples as used at the HiRISE
Operations Center.
The Mars Reconnaissance Orbiter (MRO; http://mars.jpl.nasa.gov/mro/)
High Resolution Imaging Science Experiment (HiRISE;
http://hirise.lpl.arizona.edu/) generated a large number of big
observation data products during the Primary Science Phase of the
mission - currently over 30 terabytes in over 867,000 product files -
and continues to generate products at a high rate. This has been
accomplished by using a network of automated Conductor
(http://pirlwww.lpl.arizona.edu/software/Conductor.shtml) pipelines
distributed over a cluster of 27 multi-processor compute nodes at the
HiRISE Operations Center (HiROC). When a watchdog process, that is
always running on one of these nodes, detects that new HiRISE
observation data is available from NASA's Jet Propulsion Lab (JPL;
http://www.jpl.nasa.gov/), which receives it from the spacecraft via the
Deep Space Network, the watchdog makes an entry in the sources database
table of the first Conductor pipeline segment. This initiates the data
download to HiROC that begins the sequence of linked Conductor
pipelines. Each pipeline segment defines a particular set of data
processing operations to be applied as data flows through the network of
Conductor pipelines which ultimately produce the data products that are
delivered to the science community and the public by the Planetary Data
System (PDS; http://pds.jpl.nasa.gov/) and the HiRISE web site.
The Conductor pipelines, though they operate autonomously, do require
management. For example, to keep up with the flow of new incoming data
multiple Conductors are allocated on multiple compute nodes to handle
long running procedures, such as geometric processing, thus avoiding
processing bottlenecks by processing multiple data sources in parallel.
Bad data (Deep Space Network transmission gaps in critical sections) or
problems in the underlying systems can cause processing failures that
will, if the configured failure limit is reached, cause Conductors to
stop processing the affected pipelines and notify operators to
investigate the problem before restarting the Conductors. Changes to
processing parameters can require reprocessing of some or all data
products which calls for a different network of Conductor pipelines than
is used for routine processing. Thus the data processing operators use
various Conductor networks depending on the needs of the situation. The
data processing procedures themselves are undergoing constant tuning and
enhancements that require suspending some or all of the Conductor
pipelines while new processing software and/or configuration files are
installed. These conditions call for a tool that can manage both
individual Conductors and the pipeline networks as a whole.
The Maestro package is a new addition to the Conductor software package
(http://pirl.lpl.arizona.edu/software/Conductor.shtml). It provides
remote monitoring and management of Conductor networks. The Maestro
software is based on an asynchronous, event-driven Messenger service in
which each Conductor reports its processing activities, as they occur,
to a Stage_Manager for its Theater location. Each computer system may
host multiple Theaters as needed. A Kapellmeister client can connect to
the Stage_Manager of any Theater location and request to receive a list
identifying all Conductors operating at the Theater location and
notification of changes to the list. The Kapellmeister establishes
Messenger connections, through the Stage_Manager, to the individual
Conductors to receive the notifications of all the Conductor processing
activities in real-time. The Kapellmeister provides Conductor network
managers with a graphical user interface that controls all Theater
connections, lists all the Conductors on all the Theaters with their
processing state, enables managers to send the Conductors messages to
change their processing state, and shows a matrix of all Conductor
pipelines by their Theater location with a display that summarizes all
the Conductor states. Operators can start new Conductors on any Theater
as needed as well as cause existing Conductors to safely stop processing
and, if desired, quit. The Kapellmeister can write a Profile file that
defines the current Conductor network, and can read a Profile file to
establish the defined Conductor network. Operators also have available a
new Conductor Manager interface that provides detailed monitoring and
management of all Conductor operations. The Manager may be used remotely
via a Kapellmeister or locally when running an individual Conductor.
The Maestro package - part of the PIRL Java Packages
(http://pirlwww.lpl.arizona.edu/software/PIRL_Java_Packages.shtml) -
provides a high level of view of a Conductor pipeline network combined
with detailed monitoring and control capabilities that offers
significant management effectiveness and efficiency for these networks.
|
|
Jan 20
|
|
| Ken Mighell |
NOAO |
CRBLASTER: A Fast Parallel-Processing Program for Cosmic Ray Rejection in Space-Based Observations |
|
Jan 13
|
|
| Alvaro Egaņa |
Science Data Management |
FRESSIA - A Framework for Testing Rich Applications |
|
Dec 16
|
|
| Rob Seaman |
Science Data Management |
The Art of Noise: Optimal DN encoding for CCD and CMOS detectors |
|
CCDs are linear devices. What does this mean? It does not mean that pixel
values (DNs) must be represented on a linear scale. Since CCD and CMOS
detectors are photon counting devices, they obey a Poisson (shot) noise
model. This means that such data are "heteroscedastic", with a
variance proportional to DN. Thus, a linear scaling of the data will
result in a vast oversampling of the noise for brighter pixels.
Variance stabilization techniques will be discussed that can dramatically
compress 16-bit or 32-bit integer data into a few hundred data numbers. In
combination with tiled FITS Rice compression, this produces near-optimal
encoding of astronomical data to improve both data storage and data
throughput metrics. This dual technique has been used to compress
datastreams from spacecraft, but a more formal analysis supports the
argument that a variance stabilized "Poisson encoding" is the
natural representation for CCD/CMOS data of all types. In general, the
FITS Tile Compression standard provides significant advantages over general
purpose compression formats such as the familiar gzip.
|
|
|
|
Dec 9
|
|
| Katy Garmany and Ken Mighell |
NOAO |
Using Virtual Astronomical Observatory Tools for Astronomy 101 |
|
Nov 18
|
|
| German Shumacher |
CTIO |
LSST Telescope and Site Control Software |
|
Nov 11
|
|
| Phil Daly |
NOAO |
NEWFIRM Tools and Utilities |
|
Oct 21
|
|
| Igor Suarez-Sola |
NSO |
Virtual Solar Observatory: the SDO data provider |
|
