Introduction
As never before, higher education in the United States is at a crossroads. Methods of delivery have been forever altered by the advent of widely accessible, high quality electronic communication. For-profit educational entities are challenging location-bound institutions with customer-focused educational programming. The expectations of society with regard to higher education have come to include on-demand access to quality instruction in virtually any subject, any place, at any time. At the same time, professional, state and federal bodies are placing increasingly prescriptive accountability demands on educational organizations to demonstrate results obtained with the resources provided.
Faced with this set of competing demands, organizations of higher learning are adapting by becoming learning organizations: discovering what they do best, encouraging organizational members to do their best, and continually assessing and reassessing what they do and how they can improve upon it. This last characteristic of learning organizations is referred to as a feedback loop, and it creates the basis for the ongoing work of the Office of Assessment at Georgia Tech: to facilitate the process of student and organizational learning.
Environmental Scan
What is facing the Office of Assessment at Georgia Tech?
- External Factors (external to Georgia Tech)
- Opportunities
- Board of Regents requires systematic 5-year program reviews of all academic programs; GT will need to continue to be in full compliance.
- Shift in ABET and AACSB criteria towards measurable objectives and student outcomes has pushed COE and COM to develop viable approaches to assessment.
- Benchmarking of assessment practices and findings at peer institutions will enable national comparisons of GT student learning outcomes (for example: HEDS graduate student survey; HERI faculty survey).
- Need exists for expertise in the evaluation of funded educational projects in Georgia
- Challenges
- Lack of participation in BOR committees and task forces related to accreditation, program review, and assessment.
- Little accountability for student outcomes, quality of teaching, quality of services from the BOR.
- Opportunities
- Internal Factors (within Georgia Tech)
- Strengths
- Technological cutting edge in educational programs (DSP First, etc.): brings the opportunity to provide national model for replicable engineering programs and technology-enhanced educational programming in general.
- Georgia Tech’s triennial strategic planning process allows frequent reexamination of institutional progress towards strategic goals.
- The offices of Assessment, Institutional Research and Planning, and Information Technology are all working on Institute-wide data needs, such as the GT Data Warehouse.
- Some assessment processes are already in place, due primarily to ABET requirements.
- Need exists for expertise for evaluation of funded projects within Georgia Tech.
- Weaknesses
- Lack of awareness of the essential role of assessment among faculty and many administrators.
- Faculty are rewarded for research excellence, not teaching or student success; educational research within the discipline is not rewarded in most cases.
- High degree of faculty resentment towards accountability processes.
- Lack of information infrastructure to disseminate knowledge and sharing of best practices between programs at GT and from peer institutions.
- Historical lack of centralized assessment processes and infrastructure to gather information on student performance and outcomes.
- Little accountability for student outcomes, quality of teaching, or quality of services at school, college, administrative unit, or Institute level
- Strengths
Mission
Consistent with the Institute mission of providing leadership, expecting the fullest measure of excellence, advancing innovation, providing an ethical and well-managed administration, and ensuring effective use of entrusted resources, the Office of Assessment at Georgia Tech exists to:
- conduct centralized Institute-level assessment processes, where such processes can be feasibly centralized;
- provide information and technical expertise to assist academic and other units at all levels in measuring and improving student learning and outcomes;
- facilitate continuous improvement of academic processes at GT;
- facilitate an understanding of the effect of educational technologies on student learning at GT;
- provide expertise for proposal development and evaluation expertise for funded projects internal and external to Georgia Tech on a fee basis;
- provide a focal point for the Institute’s accountability to stakeholders; and
- accumulate, generate, maintain, and disseminate institutional and benchmarking information to support assessment and general awareness of student learning and the academic experience at Georgia Tech.
Vision
The Georgia Tech Office of Assessment will be a nationally recognized leader in providing effective, current, and efficient assessment strategies and information. The Office of Assessment will also provide timely and focused methods for assessing the effect of educational technology on student learning and outcomes for all educational programs offered by the Institute, whether on-campus or otherwise.
Core Values
- As a front-line learning organization, Georgia Tech utilizes best practices and provides leadership in all aspects of its operations, and especially in academic planning and decision making.
- The citizens of Georgia and other stakeholders have the right to expect that GT will use resources in the most educationally effective and efficient manner possible.
- In our rapidly changing academic and technological environments, higher education organizations must use information about their performance to continuously improve all aspects of their operations. Those that do will be organizations that survive and prosper in the emerging global higher education community.
Goal 1
GT will be able to determine overall institutional effectiveness on an annual basis and will use the resulting information in annual and strategic planning processes.
Objective 1a: Develop and implement an annual reporting cycle for assessment activities and findings that is linked to annual reports at the school, college and Institute level.
Evaluation strategy 1a: Evaluate extent to which schools and colleges comply with annual assessment reporting requirements, and the quality of assessment information provided in those reports.
Objective 1b: Assist identified planning units in linking assessment to strategic planning documents that will ultimately roll up into Institute strategic plans.
Evaluation strategy 1b: Evaluate the extent to which assessment information has been utilized in creating strategic plans
Goal 2
GT will have in place scheduled systematic processes of assessment and renewal for all academic programs.
Objective 2a: Develop and implement a periodic, systematic program review cycle at both graduate and undergraduate levels that institutionalize assessment in academic programs, that documents the use of assessment findings for program improvements, and that provides for accountability to stakeholders.
Evaluation strategy 2a: Annually evaluate compliance with the program review cycle as given to SACS in our follow-up report. Annually evaluate the extent to which assessment is made a central feature of curriculum reviews that are performed at GT.
Goal 3
All academic units at GT will have developed assessment plans and will utilize the most effective and efficient practices and strategies in conducting assessment-related activities.
Objective 3a: The Office of Assessment will develop and disseminate planning guides for assessment and will assist academic units in developing workable assessment plans, in implementing those plans, and in interpreting resultant data as time allows.
Evaluation strategy 3a: For 1999-2000, evaluate the quality of the undergraduate assessment plans developed, especially in Management and the College of Sciences. Evaluate extent to which schools and colleges comply with annual assessment reporting requirements, and the quality of assessment information provided in those reports.
Objective 3b: The Office of Assessment will research and disseminate best practices in assessment, including assessment within the disciplines.
Evaluation strategy 3b: Track number of requests for consultation, track usage of assessment practices as evidenced in annual assessment reports.
Objective 3c: The Office of Assessment will develop and/or disseminate planning guides for the assessment and evaluation of externally funded projects, contracts, and grants, and will assist in evaluation of such projects on a time- and funding-available basis.
Evaluation strategy 3c: Track number of requests for consultation, track usage of assessment practices as evidenced in projects. Monitor time usage of assessment office personnel in evaluating funded projects
Goal 4
Academic units, funded projects, and individual faculty members will understand and employ a theory- and research-based comprehensive framework for assessing technology-enhanced educational programs and initiatives.
Objective 4a: Develop a theory- and research-based framework for assessing technology-enhanced education.
Evaluation strategy 4a: Determine effectiveness of framework through collegial critique, user survey and feedback, as well as the number of consultations provided.
Objective 4b: Develop tools and measures to assess technology-enhanced education within the framework.
Evaluation strategy 4b: Determine effectiveness of tools and measures through user survey and feedback, # of consultations
Goal 5
The Office of Assessment at Georgia Tech will provide a comprehensive information resource on assessment, and will actively disseminate assessment findings around the campus to raise the level of awareness regarding assessment issues and practices.
Objective 5a: Develop, maintain and continually expand a longitudinal data base of student assessment and performance information; use for assessment studies.
Evaluation strategy 5a: Number of studies conducted annually using this database.
Objective 5b: Develop, maintain and continually expand assessment web page, featuring in-depth information on assessment practices and a network of links to other assessment resources at GT and worldwide.
Evaluation strategy 5b: User feedback will be sought directly on the site. Comments received on web site will be collected and continually evaluated.
Objective 5c: Disseminate knowledge of how to assess technology-enhanced educational programs through assessment newsletter and through campus presentations.
Evaluation strategy 5c 1): Assess impact of newsletter through user survey every 2 years.
Evaluation strategy 5c 2): Evaluate presentations on assessing technology-enhanced education through attendee feedback.
Objective 5d: Disseminate knowledge of best practices in authentic performance assessment for direct measurement of major-specific and general education skills in the disciplines.
Evaluation strategy 5d: The number of presentations in this area will be evaluated annually. The extent to which authentic performance assessment techniques have been adopted will be evaluated as each curriculum is reviewed
Goal 6
GT will be able to ascertain the effectiveness of its general education program, of student growth and development, as well as the effectiveness of programs within each school at the graduate and undergraduate levels.
Objective 6a: Seek funding for, develop, and implement longitudinal cohort (panel) study of student performance, persistence, growth and development to assess general education.
Evaluation strategy 6a: Evaluate extent of progress on funding and implementation of study.
Objective 6b: Facilitate general education assessment across curricula in math, chemistry, physics, and oral and written communication.
Evaluation strategy 6b: Enumerate number of consultations related to each discipline; evaluate products of those consultations.
Objective 6c: Evaluate the effectiveness of educational program assessment within each discipline at the undergraduate and graduate levels via systematic program review.
Evaluation strategy 6c: Extent to which program review schedule as presented to SACS is followed; evaluate extent to which assessment findings guide curriculum review and are used as the basis for curriculum planning and renewal, as evidenced in program review follow-up action plans
Goal 7
Consistent with providing high-quality, usable information at the unit level, GT will use economies of scale in applying assessment practices.
Objective 7a: Centralize mechanisms for assessment data collection, analysis and dissemination as it applies to survey research.
Evaluation strategy 7a: Extent to which entering student, graduating senior, graduate student exit, alumni, employer, and continuing student surveys have been implemented on a systematic, centralized basis.
Objective 7b: Develop and implement common methodologies for assessing the impact of technology-enhanced educational initiatives.
Evaluation strategy 7b: Extent to which common methodologies are being utilized to assess the impact of technology-enhanced educational initiatives. Determine through user survey and number of consultations
Goal 8
Georgia Tech will meet and exceed student outcomes-based assessment and institutional effectiveness criteria in regional and professional accreditation reviews.
Objective 8a: Faculty and staff at GT will be informed and involved in institutional self-studies.
Evaluation strategy 8a: Extent of representative makeup of committees charged with examining Georgia Tech’s compliance with each area of the SACS criteria.
Objective 8b: All academic and administrative units will meet and exceed planning and evaluation criteria promulgated by SACS on a continuous basis.
Evaluation Strategy 8b 1): As part of the next SACS self-study, the appropriate committee will evaluate the extent to which GT is in compliance with planning and evaluation criteria.
Evaluation Strategy 8b 2): The Office of Assessment will review assessment and evaluation plans as submitted, and will review annual assessment reports from all units.
Objective 8c: All colleges and schools within colleges will meet and exceed planning and outcomes assessment criteria promulgated by relevant professional societies or professional accreditation boards (e.g., ABET, AACSB).
Evaluation Strategy 8c: As requested, the Office of Assessment will review assessment and evaluation plans, and will consult with units on approaches to assessment. Annual assessment reports will be compared with assessment plans as submitted, and differences will be noted.
Objective 8d: As a whole, the Institute will demonstrate substantial compliance with all academic aspects of the SACS Institutional Effectiveness and Planning criteria, and will do so on a continuous basis.
Evaluation Strategy 8d: 5-year program reviews and assessment reports in annual plans will be used to evaluate the extent to which colleges and schools are in compliance with accreditation criteria. Annual institutional effectiveness report will be used to highlight trends in accreditation standard compliance
Goal 9
The Office of Assessment will provide timely and effective assessment methods and strategies, as requested, for evaluating asynchronous and other off-campus programs.
Objective 9a: The Office of Assessment will use passive, web-based data collection strategies effectively for collecting assessment data on web-based courses and curricula.
Evaluation Strategy 9a: Evaluate extent of utilization, number of courses assessed using these tools, extent of data captured and problems encountered.
Objective 9b: The Office of Assessment will use active assessment strategies effectively to assess distance or off-campus programs.
Evaluation Strategy 9b: Evaluate response rates to web- or paper-and-pencil surveys; evaluate quality and usefulness of data obtained through assessment of web interaction, interview, focus group, test, presentation, or other learning activity. Evaluate response and participation rates to on-campus vs. off-campus assessments.
Goal 10
The GT Office of Assessment will provide expertise in developing evaluation sections of funded proposals from faculty at GT, will provide evaluation of funded projects within and without GT on a fee basis, and will become a recognized leader in funded program evaluation within the State of Georgia.
Objective 10a: The Office of Assessment will provide proposal development expertise.
Evaluation Strategy 10a: Number of proposals on which the office assists each year; number of successful proposals.
Objective 10b: The Office of Assessment will provide evaluation of funded projects within and without GT.
Evaluation Strategy 10b: Number of funded projects under contract; dollar volume generated; number of projects evaluated internally/externally; topical and geographic dispersion of funded evaluations.
Goal 11
The GT Office of Assessment will operate in an effective, efficient, and customer-focused manner, utilizing best practices in assessment in all cases.
Objective 11a: The Office of Assessment will be periodically reviewed.
Evaluation Strategy 11a: Every five years, the Office of Assessment will be reviewed by external reviewers, and will be judged to be operating in an effective, efficient, and customer-focused manner.
Objective 11b: The Office of Assessment will meet and exceed user needs for assessment information.
Evaluation Strategy 11b: In an annual analysis of structured feedback from users, the GT Office of Assessment will be judged as providing quality information and timely analyses of relevant assessment issues at GT.