Skip to end of metadata
Go to start of metadata

General Description

"General Description"

Icon

The Description header should provide a specific definition and use of the service or system. This should be short and to the point.

This group is interested in exposing data generated by activities within the LMS for a variety of purposes: early warning, student jeopardy, retention, course evaluations, etc.

Stakeholders

"Stakeholders"

Icon

The Stakeholders sub-header should provide a specific persons involved with the service or system, including the Business Owner (The person responsible for the business case for the service or system.), Product Owner (The person responsible for the functional requirements.), the Technical Owner (The person responsible for the administration of the service or system.).

Other Participants

Email group

Upcoming Activities

Icon

Report Modification (12/7/2010)
Modifications to the existing reports where suggested by Emmanuel College and would like to be considered for adoption/production. Please review these reports to see if the new format meets your needs to ascertain if the modifications can replace or enhance the current reports, or if they will need to be separate.


Call with John Campbell, Purdue University
John has been using data from Vista quite successfully to identify students who were getting in trouble in courses. He has offered to put his Vista admin in touch with this group to talk about the practical aspects of getting at data.
Date/Time: TBD
Location: TBD
NOTE: Other than an initial phone call, we have yet to make contact with John Campbell to set up a time to discuss Purdue's efforts.


Service Levels

"Service Levels"

Icon

The Service Levels sub-header should provide specific support levels for both technical and end-user support. In addition, performance and availability thresholds should be included as well as the metrics for assessment.

At the same time, several other campuses contacted UMassOnline expressing similar interests in developing reports based on data captured in Vista. These folks (MCLA, WIT and Emmanuel) got together on a phone call to better understand what campuses are looking for (both in reporting and access) and see if all involved could organize ourselves in order to determine if there might be common needs, output/formatting, reporting etc. Some campuses were looking for very basic dumps of raw data and others appeared to want more sophisticated reports. The first effort was to try and understand the variety of requests so efforts could more forward appropriately representing the broadest needs.

Training

Technical Training

End-user Training

End-User Support

Technical Support

Availability

Performance

Background

In the early summer of 2010, Peter Allmaker contacted UMassOnline to ask a few questions about getting access to Vista data. Subsequent discussions revealed that UMassOnline does not have direct access to Vista data, however can query through some of the native Vista reporting tools as well as some legacy systems developed by UITS.

July 2012: With the UMOL migration to Learn some interesting developments have arrisen around Data Access and Analytics. Blackboard has developed a product that would tie into both Campus SIS systems and the Bb Learn 9.1 system by creating a data-mart of these data sets. Blackboard Analytics for Learn™

During BbWorld 2012 Blackboard demonstrated this system and it is believed that the data and reports that are made available would be very useful to both UMassOnline and the campuses hosted by UMassOnline and would far exceed anything UMOL could potentially setup alone.

Also, in recent weeks, there has been interest from hosted campuses to access the database for Bb Learn. While Blackboard now publishes it's "Open" schema for the database, this doesn't necessarilly mean that the database is "open". BBLEARN-422   BBLEARN-424  By providing campus administrators direct access to the database it would expose them to campus data that those campuses may not want them accessing.

Related Operations, Services & Systems

"Related Projects"

Icon

The Related Projects header should provide links to other services and systems that this service/system depends on, or services/systems that rely on this service or system.

Specifications

"Specifications"

Icon

The Specifications header should provide information about the currently running version of the service or system, dependencies (what other services and systems might be affected by or affect this service or system), and other technical issues.

UMassOnline provides LMS data to campuses through three reports:

Our understanding of both our access to Vista data and what is possible includes:

  1. Not all of the data generated in the use of Vista can be accessed. Some Vista (or supporting) packages (tables, fields, elements) with specific data is not accessible.
    • The "package" that UMassOnline tracking reports are pulled from is a Cognos package that was built off Vista tracking tables. UMassOnline is not sure if all the fields in the Vista tables were included in this package. So there maybe Vista data that is not available and there maybe some available to those who understands and can access the Vista tables.
  2. Some of the open packages' schema are known to us and others are not.
    • Same as noted above.
  3. The reports UMassOnline runs hit the packages that are both open and that we understand the schema for.
  4. UmassOnline can provide campuses access to the data fields for only the open packages (that Vista lets us see) and in which we have a schema (where we understand the table headers).
    • These are expressed in the Cognos tracking package.
  5. There are other data that is available, however we do not know the schema and thus cannot provide the data fields to the campuses.
    • In addition, there is other data that could be available if UmassOnline reworked the Cognos package and based it on data in Vista tables. It is this data however that UMassOnline has difficulty with.

Related Use/Support Policies

"Related Use/Support Policies"

Icon

The Related Use/Support Policies header should provide information regarding specific policies related to the service or system.

Known Issues

"Known Issues"

Icon

This header should be used to list any issues associated with the service or system. Issues might include: questions or discussions underway regarding the service/system or open/closed tickets (see below), etc..

Tasks

"Tickets"

Icon

This header should be used to list any tickets associated with the service or system. Issues might include, performance, bugs, or unmet functional or technical requirements.

0%

Task List

  1. handler

    Review suggested modifications to "Tool Usage" "User Session Count" and "Course Item Usage" reports

    Priority MEDIUM
    pmasson
    Dec 07, 2010

Status

"Status"

Icon

The Status Header should be used to indicate any activity associated with the development, administration, support or extension of this service or system. In addition, the "Excerpt Include" Macro should be placed here in order to push status updates into the various project tracking pages in Confluence. Note, you can use this same information in the Comment field when editing the page.

Emmanuel College has evaluated their reporting needs and defined the following criteria for a set of weekly reports. These reports are based on the currently available reports provided by UMassOnline (Tim Lambert) for Tool Usage, User Session Count and Course Item Usage reports. The report name if followed by the suggested modifications including a production schedule.

Most of the proposed changes are substitutions such as swapping out ID’s for sections and users for Name, Title, Login, etc. The most notable change would be the inclusion of collecting and reporting on faculty activity in some of the reports.

Tool Usage

  • Do not need PATH
  • Source_ID – need instead user unique ID and/or log in ID
  • Learning_Context_ID – need instead Section ID (ie. HRM_9028_OL_102F)
  • Remove the Syllabus tool
  • Data from Section IDs of “OL” courses (we will send you a list of the specific Section IDs we want data pulled from)
  • One report for all Section IDs
  • Reports on every user - instructor, designer, and student
  • Send weekly data every Monday AM

User Session Count

  • Do not need PATH
  • Source_ID – need instead user unique ID and/or log in ID
  • Learning_Context_ID – need instead Section ID (ie. HRM_9028_OL_102F)
  • Data from Section IDs of “OL” courses (we will send you a list of the specific Section IDs we want data pulled from)
  • One report for all Section IDs
  • Reports on every user - instructor, designer, and student
  • Send weekly data every Monday AM

Course Item Usage

  • Do not need PATH
  • Do not need Source_ID
  • Learning_Context_ID – need instead Section ID (ie. HRM_9028_OL_102F)
  • Need aggregate data based on Section ID (not per Source_ID)
  • Data from Section IDs of “OL” courses (we will send you a list of the specific Section IDs we want data pulled from)
  • All Section IDs in one report
  • Send weekly data every Monday AM

7 Comments

  1. The suggested column modifications would suit many of my needs.  I would suggest that establishing a filter for certain courses or tools could be handled by the institution after the data is recieved. That might simplify report generation somewhat and it would also separate providing the data, something UMOL has to help us with, from processing the data, something we will each be doing differently anyway.   Implicit in the discussion is a filter by institution, of course.  A starting point for that might be to use the existing backup filter which already selects for 'current' whatever that means to each of us.

    The suggested weekly reporting frequency is the coarsest I would want to see.  For lack of a daily version I'd like to see a date/time of last access column added somewhere, maybe in the user session count report. 

    And since I will be automating processing I'd want the report to be machine readable (csv or something of the sort).  I'd also wish for it to be machine fetchable.  Maybe dump a file into our existing SFTP space?  

    Thanks Emmanuel for putting this on the table. 

    1. I'll look into a last access column. Perhaps there is something I can use in the user session sheet, however these three reports on tracking are generally geared toward the actual sessions, tools and course item usage and are not student specific.

      I think part of the intention here seems to be that folks are looking for data for specific users and not the features and sessions that these reports are actually generating.

      As long as we can get the data you need, I don't see a problem arranging for csv output onto an sftp server. Although, the sftp host is likely not going to be the same host that folks use for batching.

  2. A couple of notes for consideration on technical implications of running the suggested reports:

    • The existing reports that are/were used to generate the types of data potentially available for tracking purposes would need a redesign. Currently they are set to query the semester (e.g. Fall, Spring, Summer) and the year (e.g. 2010) in the path of the learning context. This suited the needs of the campus who initially contracted with UMOL to generate these reports. It would need to be adjusted for something more generic as no school interested in this type of data uses the same format.

    With regard to the specific fields highlighted by Emmanuel, see my notes below.

    Tool Usage

    • Do not need PATH
      • (Fine)
    • Source_ID – need instead user unique ID and/or log in ID
      • Source_ID is the same at Section ID or UNIQUE_ID or sourcedid.id.
      • The TOOL_NAME does a count of the times the tool was used in a specific section/Learning Context. It has not been designed to list the same tool individually while listing the User ID who used it. Don't forget this is a report on the tools usage not user behavior.
    • Learning_Context_ID – need instead Section ID (ie. HRM_9028_OL_102F)
      • Same as first point above.
    • Remove the Syllabus tool
      • It shouldn't be too hard to filter out this tool. The reports look for all tool usage so a filter would need to be implemented.
    • Data from Section IDs of “OL” courses (we will send you a list of the specific Section IDs we want data pulled from)
      • A significant step in generating these reports is the need to identify the potentially active sections where tracking data for live sections maybe found. If campuses provided a list of the sections they are looking for tracking data for, then this step would be eliminated and would therefore make the process slightly simpler.
    • One report for all Section IDs
      • I'm not clear on this request. All section IDs that are identified would be included in the report.
    • Reports on every user - instructor, designer, and student
      • Again, this report is based on reporting to the TOOLs used and not who is using them. It is my understanding that the count in this report is counting ALL use of a tool regardless of role.
    • Send weekly data every Monday AM
      • To pull all nine of the available tracking reports will vary on the size of the institution and how many sections are offered, but has run between 30 to 60 hours in the past. Based on the low 30 hours, if we were to pull the three reports requested, we can expect a complete run time of about 13 hours. Granted, if we were to somehow rework and schedule reports weekly and find a way to pull the last 7 days only rather than based on specific dates, the run times and load on the system might be minimized.

    The same Tools comment apply for User Session and Course Item Usage reporting.

    I have had off the record discussions with the UITS Cognos group and because of the nature of the back end reporting system and the need to run against the production Db of Vista for most recent and accurate data, it has been suggested that a more efficient way of reporting would be to work with them on building out a data mart based on Vista data and then reporting off that. Perhaps we should be considering contracting with that group to develop a data mart and roll these tracking conversations into that.

  3. I received a phone call from Blackboard about my request for more information on Blackboard Analytics. Should I respond and have them contact UMOL for a demonstration?

    1. Absolutely. I know quite a few people here at UMass Boston who are interested.

  4. Much of the information on this page seems tied to Vista. Would it make sense to have a separate or sub-page for BB Learn?

    1. I think as we are all in the process of migrating to Learn and how interest seemed to drop off for Vista we can continue to adapt and develop this NIFTI discussion for Bb Learn.