Open Source “Digital Library” Systems: Streetprint & Greenstone
An introduction and comparison
Prepared by Martha Chantiny,
Head, Desktop Network Services
114th Annual ASEE Conference Workshop 2 University of Hawaii at Manoa, Hamilton Library
June 24, 2007
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
•NOT a complex Content Management System
•Intended for the one-person project/organization
•Does not require technological or programming wizard level skills
•Mostly platform agnostic
Systems to make digital collections available, searchable & harvestable
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
History http://wiki.greenstone.org/wiki/gsdoc/others/Greenstone_history.htm
Greenstone – created 1997-98 Greenstone software grew out of the New Zealand
Digital Library Project at the University of Waikato Developed and distributed in cooperation with
UNESCO Licensed under the Gnu Public License (GPL) First CD-ROM the Humanity Development Library
1.3 issued in April 1998
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Historyhttp://www.crcstudio.arts.ualberta.ca/streetprintorg/streetprintMLA.pdf
Streetprint – created 2003 Grew out of Canada Research Chair Humanities Computing
Studio project to create “computerized and integrated multi-media humanities research centre to unite humanities research and digital technology “
Funded by Canada Research Chairs Infrastructure Fund $54,860 in June 2001
Licensed under the Gnu Public License (GPL) First collection was digitized British street literature
Streetprint: Revolution and Romanticism
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Pros & Cons
Greenstone CD runtime output Any metadata scheme Cross-collection searching Large user base, active
listserv Multi-lingual Highly scaleable
Streetprint CSS & templates
provided PHP (easy to change
functionality) Comment feature Easy to use
FEATURES
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Greenstone Complex mix of
specialized code – more difficult to understand - mix of GUI and coding
Requires more SysAdmin level knowledge & skills
Streetprint No batch load Code not bug free or
finished Only Dublin Core No cross-collection search Small user base, inactive
lists How scaleable?
Pros & Cons DRAWBACKS
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Installation - Streetprint
1. A server (Linux or Mac with OSX or a Windows server) 2. Web server software (e.g. Apache)3. MySQL version 3.23 or higher 4. PHP version 4.1 or higher5. Know how to download & expand tar.gz or zip files6. Knowledge of (or access to someone who can) change permissions to allow user running apache to write to the Streetprint directory, create databases, & assign rights in MySQL, configure web server software for PHP7. Desktop computer with any web browser
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Installation - Greenstone
1. A server (Linux or Mac with OSX or a Windows server) 2. Web server software (e.g. Apache)3. PERL/CGI 4. GCC Gnu C++ compiler and GDBM Gnu database manager compiler (if the precompiled binaries don’t work for you)
5. Knowledge of how to download and correctly expand tar.gz or zip files6. Ability to (or access to someone who can) change web server config & cgi-bin config files; add users7. Desktop computer with Java Runtime Environment software, installed GLI java applet and web browser
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Setting up a collection –Streetprint Database creation
Geek Alert! This chapter of the Streetprint Engine manual is by far the geekiest of the lot. If you need any help with this section, we recommend you consult your nearest internet wizard or technical support staff member.
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Setting up a collection –Streetprint Name the DB in MySQL
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Setting up a collection –Streetprint
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Setting up a collection –Streetprint User profiles
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Setting up a collection –Streetprint User profiles
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Setting up a collection - Greenstone
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Setting up a collection - Greenstone User Accounts
Info only Editable
Info only
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Setting up a collection - Greenstone User Accounts
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Setting up a collection –Greenstone Database creation
The installation begins by asking which directory to install in. By default, all Greenstone software is installed in the directory /usr/local/gsdl if it is the root user who is doing the installation. Next you choose to install binaries or source code. You are then asked some questions about your webserver setup. A valid cgi executable directory (normally called “cgi-bin” on Unix systems) is required and you need to enter the web address of the cgi directory. A directory called “collect” will be created during the install process.
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Setting up a collection –Greenstone Database creation
“Greenstone Librarian Interface” or GLI applet should be installed at this point
--New Collection--
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Setting up a collection – GreenstoneDescription, Contact, About
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Setting up a collection – StreetprintDescription, Contact, About
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Setting up a collection – StreetprintDescription, Contact, About
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Adding Data - Streetprint
Important note: By default, most PHP servers limit uploaded file sizes to 2MB or smaller. If you plan to add media files which are larger than 2MB, you may need to contact your server administrator and have them change this setting first.
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Adding Data - StreetprintDocument type
This can be used to distinguish items in the collection, e.g. different report series, or different sizes
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Adding Data - StreetprintDefining types
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Adding Data - StreetprintDefining types
Category = Subject Heading
By default this creates a pull-down list in the Add/Edit mode
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Adding Data - StreetprintMetadata
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Adding Data – GreenstonePlug-ins (*.pm)
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Adding Data – Greenstonemore about plug-ins
Two types, metadata and document plugins to process files, some •group several files into one document, •some split one file into several documents, and •some have a one to one mapping.
ImagePlug •uses the imagemagic utilities to •create derivatives(thumbnail images) and, •extract image metadata (width height format)
ImagePlug can easily be extended to extract more extensive image metadata if required
From the documentation:
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Greenstone customizationPlug-ins
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Adding Data – Greenstoneconfiguring plug-ins
I am building a collection that is mainly JPEG files, and 1 PDF file of a pamphlet. Is it possible to type a description page in MS Word for each JPEG in the collection, and associate the word document with the JPEG so that when the JPEG image is retrieved, the user also has the option of clicking an icon and reading MS Word document description of the image.
Do you want the text in the Word document to be indexed? What you could try is naming your documents like photo1.jpg, photo1.doci.e. same filename for word doc and image doc. Then have WordPlug and ImagePlug in the collection. On one of the plugins use the -associate-ext option - if you add it to WordPlug, set the value of the option to jpg. If you use the option on IMagePlug, set the value to doc. This will mean that one version is processed as a document, the other is associated with it. Which way round you do this affects the processing of the files. (whether you get the word document indexed or not, whether you get the image processed into thumbnails etc) You'll also need to play around with format statements.
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Diverging Paths
At this point using the 2 systems involves very different workflowsGreenstone = quasi-batchStreetprint = record by record
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Input - Greenstone
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Input – Greenstonedrag & drop all file types
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Input – Streetprintdefault form for metadata entry
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Input – Streetprinterror checking & publish
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Input – Streetprintadding images
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Input – Streetprintadding images
A minimum of two files per record need to be pre-processed and uploaded
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Input – Streetprintadding full text
Cut and paste (or type in!?) only plain ASCII
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Input – Streetprintadding media
Unlike images – media files must be uploaded from the PC
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Adding Data – Greenstonebuild
After adding files …build collection
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Adding Data – Greenstoneadding & editing metadata
If you haven’t carefully manipulated and coded your files, the extracted metadata isn’t always very useful
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Break?
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Searching - Greenstonesetup
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Searching - Greenstonesetup
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Searching - Greenstoneuser interface
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Searching - GreenstonePreferences
What the end user sees:
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Searching - GreenstonePreferences
# page content##############################################################_pagetitle_ {_collectionname_: _textpreferences_}_imagethispage_ {_textpreferences_}_htmloptions_ {}_htmloptionson_ {<p><tr>_fcoption_</tr><p><tr>_hloption_</tr><p><tr>_extlinkoption_</tr>_If_(_PreferenceDocsFromWeb_,<p><tr>_intlinkoption_</tr>)}_fcoption_ {<td valign=baseline>Document page layout</td><td><input type=radio name=fc value=1 onClick="updatefc(1);" _If_(_cgiargfc_, checked)>navigation bar at top<br> <input type=radio name=fc value=0 onClick="updatefc(0);" _If_(_cgiargfc_,, checked)>no navigation bar</td>}
Functionality and look of page controlled in prefs.dm macro file:
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Searching - Streetprintsetup
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Searching - Streetprintuser interface
Default is ultra-simple keyword search
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Searching - Streetprintuser interface
Advanced search with instructions or multiple pull down menus
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Streetprint - user interfaceComments
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Customizing
Which code looks like more fun?
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Customizing
format VList <td valign=top>[link][icon][/link]</td><td valign=top>[highlight]{Or}{[Title],Untitled}[/highlight]<i><small>{If}{[Date],<br>_textdate_[Date]}{If}{[NumPages],<br>_textnumpages_[NumPages]}{If}{[Source],<br>_texts\source_[Source]}</small></i></td>
$tmpl->addVars( "viewtext", array( "TEXT_NAV" => $textnav->getHtml(), "IMAGE1" => $image1, "MAIN_IMAGE_WIDTH" => $mainImgWidth, "TEXT_TITLE" => $text->title, "TEXT_TITLE_UC" => strtoupper($text->title), "RECORD" => $text->getRecord($gfx), "RIGHT_COL_WIDTH" => $rightColWidth,));
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Customizing
$tmpl->addVars( "searchresults", array("THUMBS" => $thumbnails, "PAGINATION" => $pag, "VIEW_SWITCHER" => $switcher));if(!empty($legend)){$tmpl->addVar("commentLegend","COMMENT_LEGEND", $legend); } else { $tmpl->setAttribute("commentLegend","visibility","hidden"); }
<format><td valign=top>[link][icon][/link]</td><td valign=top>[srclink][srcicon][/srclink]</td><td valign=top>[highlight]{Or}{[ dls.Title],Untitled}[/highlight]<i><small>{If}{[Date],<br>{If}{[NumPages],<br>_textnumpages_[NumPages]}{If}{[Source]</small></i></td> </format>
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Streetprint customizationTemplates, Pages, Includes
/streetprint/DB/include/•about_navigation.inc•allObjects.inc•navigation.inc•sp_fieldnames.inc•sp_footer.inc•sp_graphicfiles.inc•sp_header.inc•sp_initialization.inc•sp_subdir_header.inc
/streetprint/DB/Pages•about_contact.php•about.php•advancedsearch.php•basicsearch.php•browseby.php•browselist.php•browse.php•bytitle.php•search.php•searchresults.php•viewfulltext.php•viewimage.php•viewmedia.php•viewtext.php
/streetprint/DBs/templates/Pages/ •about_tmpl.html•advancedsearch_tmpl.html•basicsearch_tmpl.html•browseby_tmpl.html•browselist_tmpl.html•browse_tmpl.html•index_tmpl.html•searchresults_tmpl.html•viewfulltext_tmpl.html•viewimage_tmpl.html•viewmedia_tmpl.html•viewtext_tmpl.html
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Greenstone customizationMacros,Config files
/gsdl/macros•about.dm•browse.dm•collect.dm•dateqry.dm•extra.dm•gli.dm•help.dm•home.dm•html.dm•nav_css.dm•pref.dm•query.dm•style.dm•yourhome.dm
There are 103 macro files, most of them are for language definitions: arabic to urdu
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Greenstone customizationFormat statements & string items
[link][/link][srclink][/srclink][icon][srcicon][num][numleafdocs][Text][RelatedDocuments][highlight][/highlight][Summary][DocOID][DocRank][DocImage][collection:meta-name]
VListHListDateListSearchVListDocumentVListCL1VListCL1HListCL1DateList
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Greenstone customizationFormat statements & string items
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Greenstone customizationCGI
cgi arguments used by library encode the parameters of each page, e.g., what kind of page to display, what query to carry out. a is the main cgi argument and determines which action is used to display the page.
1 a=p: Page action 2 a=q: Query action 3 a=d: Document action 4 a=extlink: External Link action
Queries then have many options:
t search type - if b=0, 0=all, 1=some. if b=1, 0=boolean, 1=ranked
m max number of documents to return in search results
o how many documents to display per page of search results
r which result number to start from for the current page of results
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
OK, have we had enough tech?
Time for a tour …
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Sites using Greenstone
http://nzdl.sadl.uleth.ca/cgi-bin/library
http://www.booksfromthepast.org/
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Sites using Greenstone
http://image2.nybg.org/cgi-bin/nybg.exe
http://bridges.lib.lehigh.edu/cgi-bin/library?a=p&p=home
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Sites using Greenstonehttp://nupepa.org/Hawaiian Language Newspapers
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Sites using GreenstoneHawaiian Language Newspapers part of The Hawaiian Electronic Library http://www.ulukau.org/
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Sites that use Streetprint
http://www.opmagazine.com/streetprint/
http://www.infoapolice.sk/streetprint/
http://cushing.tamu.edu/collections/images/
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Sites that use StreetprintUniversity of Victoria Maltwood Art Museum and Gallery
http://maltwood.uvic.ca/physics/
http://www.maltwood.uvic.ca/lorenzen/
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Sites that use Streetprint
http://www.spleencoffin.com/streetprint/
http://www.edmontonsmallpress.org/catalogue/
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Greenstone at UH Manoa: not as far along as we thought we’d be
http://128.171.57.25/gsdl/cgi-bin/library
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Greenstone at UH Manoa:HSPA
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Greenstone at UH Manoa:HSPA records - default
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Greenstone at UH Manoa:HSPA - labelled
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Greenstone at UH Manoa:HSPA search types
Sugar CompanyContent type Keywords
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Greenstone at UH Manoa:Building a Rainbow
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Greenstone at UH Manoa:Trust Territory of the Pacific Archive
Photoshttp://libweb.hawaii.edu/digicoll/ttp/ttpi.html
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Greenstone at UH Manoa:Trust Territory of the Pacific
Archive Photos
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Streetprint at UH Manoahttp://digicoll.manoa.hawaii.edu/
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Streetprint at UH ManoaSocial Movements
Converted from a home-made MySQL implementation
http://digicoll.manoa.hawaii.edu/socmovements/
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Streetprint at UH ManoaKrauss Index
Being input one by one from 3x5 index cards. Eventual goal is to link each card to an image of the article to which it refers.
http://digicoll.manoa.hawaii.edu/krauss/
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Streetprint at UH ManoaSave Our Surf
Scanned, OCR’d & completed over two semesters by one student assistant
http://digicoll.manoa.hawaii.edu/sos/
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Streetprint at UH ManoaDemonstration site - TRAIL
Created as a “proof of concept” to present to GWLA Directors with a proposal to fund a multi-year digitization effort
http://digicoll.manoa.hawaii.edu/techreports/
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Streetprint at UH ManoaDemo site - Hawaiian Music
http://digicoll.manoa.hawaii.edu/music/
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Streetprint at UH ManoaSteve Thomas Traditional Navigation
Scanned & proofed by two students in just a little over a year. Using comment feature to obtain expanded description info
http://digicoll.manoa.hawaii.edu/satawal/
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Links
•New Zealand Digital Library http://nzdl.sadl.uleth.ca/cgi-bin/library
•Greenstone http://www.greenstone.org/
•Greenstone Wiki http://wiki.greenstone.org/wiki/index.php/GreenstoneWiki•Streetprint http://www.crcstudio.arts.ualberta.ca/streetprintorg/ (http://www.streetprint.org)•Streetprint Sourceforge download https://sourceforge.net/projects/streetprint/
114th Annual ASEE Conference Workshop 2 – University of Hawaii at Manoa, Hamilton Library, June 24, 2007
Open Source “Digital Library” Systems: Streetprint & Greenstone
Presentation available online: http://www2.hawaii.edu/~chantiny/ASEE.ppt
Questions? Comments?
