National Science Foundation

Program Effectiveness Review

Common Knowledge: Pittsburgh

Presentation

March 20, 1997

Questions and Indicators

1. Who is the primary audience for your project work?

Students and staff in the Pittsburgh Public Schools

a. What is the most important effect you've had on your primary audience so far?

CK:P has fostered the constructive integration of instructional technology into the everyday life of teachers and students within the PPS. Staff and students communicate via email, research topics on the WWW, use newsgroups to access online expertise and modify their approach to teaching and learning. In addition, a large number of staff dedicate personal time and finances to the Internet in order to learn about its potential in their classrooms.

b. In what time frame will your work have its full impact, and on whom?

CK:P's network topology and site servers provide full Internet access to staff and students at CK:P schools. At non-CK:P schools within the district, staff has access through central servers and the district modem pool, but students do not. Presently, there are over 3000 staff (approximately 60%) with email and home access through CK:P. Over 1500 students at CK:P sites have such accounts. By the end of the project there will be 29 schools (approximately 33%) with LAN/MAN/WAN access through the project. Over 2/3 of the district's schools have access to the Internet through their library via modem. This will have taken 5 years to accomplish - 2 in a research mode and 3 in a production mode. Full impact will occur not only when all students, staff and schools have access to network resources, but they integrate them into student learning opportunities and professional development. This will occur if the District passes its technology plan - a plan which builds on CK:P's research. The time frame for full impact could be from 3 to 5 years.

2. What specific measures do you use as indicators that your work is having an impact?

Measure of use include:

• Number of accounts.
• Log-ins by teachers and students and growth in this over time.
• An on-line user survey which collects data from all teachers and librarians with accounts about the extent and nature of their Internet use.
• An on-line user survey that suggested teachers were spending their own money.

Measures of the consequences of use include:

• Analysis of RFPs examining how teachers intend to change classroom activities through Internet use.
• Teacher self-reports in both in-depth qualitative interviews and the above mentioned survey responses that ask about impact on teachers, students, and classroom functioning after they have had one or two years experience with it.
• Student self reports (survey and in-depth interviews) on the impact on teachers, themselves, and the classroom.

We have evidence that suggests the following outcomes:

• enhanced professional development for teachers including the development of new roles for them such as grant writer, paper presenter, taking on-line courses, Increment Credit course enrollment, joining teaching discussion groups.
• increased motivation for students.
• changes in classroom delivery based upon access to human and data resources never before available.
• broadening of students' perspectives through interactions with those they would not normally interact with.

a. What is unique about your work, relative to others in the field?

Common Knowledge has a number of components that make it unique in the field of applying wide-area networks:

• a process orientation,
• working with an entire urban school district and measuring its success based on change within the district as a whole,
• day to day project and site management by school teachers,
• a technical architecture that is distributed and easily scaled,
• qualitative evaluation focused on process which integrates quantitative components at strategic points,
• full Internet access for students,
• research in a real-world setting.

b. If your work had not been funded by NSF how would science education been diminished?

With funding from NSF, a conceptual framework provided by Dr. Robert Carlitz, and the expertise brought about by the collaboration with the University of Pittsburgh, the Pittsburgh Public Schools and the Pittsburgh Supercomputing Center, CK:P was able to build both the technical and human infrastructure to make the project succeed. Without funding for research and implementation, the district would not be able to put the appropriate infrastructure in place. At best, technology and WAN access would have been purchased without the human support necessary to build expertise throughout the school district.

With respect to science education, sites such as Westinghouse High School, Phillips Elementary School and East Hills International Studies Academy would not have access to online science databases, projects such as "The Winter Solstice Project" and online mentors.

In particular the Science and Mathematics (SAM) program at Westinghouse H.S. (whose student body is 100% African-American) has benefitted greatly from this grant. Although the SAM program existed prior to our project, it was difficult for students and staff to get expert advice and support for their research. In addition, there was no way to publish their research publically and obtain feedback. It was extremely difficult to connect the work of the students and staff with the professional science community. CK:P also served as a lever for reform at Westinghouse which led to cross-age tutoring, team teaching and modifications of the master schedule.

3. What is the relationship of your work to systemic education reform?

In order for the CK:P model to be considered systemic reform, it must:

• provide individual access to the Internet for all staff and students,
• develop support mechanisms that provide technical and educational help for users,
• implement a human and technical architecture that is scalable and sustainable,
• integrate with the district's overall strategic plan,
• change the culture - language, behaviors, and beliefs about the use of technology - in the district,
• demonstrate growth in communication, writing, problem solving, and research skills that are consistent with national standards in mathematics, science and technology.

The ultimate goal of Common Knowledge is to implement network technologies in all schools in the district.

a. How does your work use/promote content and performance standards in mathematics or science?

Use of the Internet is consistent with national, state and local standards regarding communication skills, problem solving skills, interdisciplinary learning, and technological literacy. These skills are consistent with standards created by both the National Council of Teachers of Mathematics and the National Council of Science Teachers. These skill are also reflected in the Pittsburgh Public Schools' Core Curriculum Frameworks. Specifically, through the RFP process, CK:P chooses "reform-oriented" projects at the classroom level and supports these projects by implementing necessary technology. CK:P also provided a great deal of input into the district's technology plan. This input was consistent with best practices in the professional literature.

b. In general, what is the best use of the technology central to your work in support of higher standards?

Use of the Internet provides students and staff with an authentic context for communication, data collection and research. CK:P has found that students and staff take a higher degree of ownership and pride in their work when it is publically available to the Internet community.

4. Have other NSF funded projects/programs aided your work?

A number of NSF projects have provided support for our work.

a. Which have been most influential on you and why?

The Pittsburgh Supercomputing Center, an NSF funded project, has been most influential in providing technical expertise to Common Knowledge.

One of our schools, Westinghouse High School, has a collaboration with the CoViz project. Teachers and students are using CoVIS software and CK:P connectivity to participate in collaborative science research.

CK:P works closely with the Mathematics Forum (http://forum.swarthmore.edu) at Swarthmore College. We use their online materials in our workshops. We also have co-presented at a number of Pennsylvania educational conferences.

CK:P has a close working relationship with the PUMP Algebra project. We are presently putting a LAN in at a PUMP site. Teachers in the PUMP program hope to use the Internet in their curriculum. CK:P provides an online mailing list for PUMP teachers

(wg-pump@pps.pgh.pa.us).

b. How have you connected your efforts with specific NSF projects/programs to reach a larger audience, support other reform activities, etc.?

We attempt to reach a larger audience through our online materials (http://www.info-ren.org/projects/ckp). We disseminate information at NSF sponsored conferences, professional meetings such as NCTM, AERA, APA and NCSM conferences, collaborating with other projects such as the Mathematics Forum at Swarthmore, and speaking locally in Western Pennsylvania.

5. How do you see your work being implemented on a larger scale, or your ideas widely disseminated to others?

We see the most positive aspect of our project being the collaborations and processes that lead to school reform. The model for collaboration includes universities, school districts, community centers, city government, and libraries. The collaborations allow projects to maintain both their research and production capabilities. The processes - supporting teachers, local Requests For Proposals (RFP), migration activities, production activities - allow projects to be sustainable, scalable, and supportable. Although we have attempted to disseminate information through traditional means - conferences, papers, publications, online resources - we are still not satisfied with this form of transfer.

a. How should this occur, and if not by you then whom?

We would like to offer internships to interested school districts. These internships would provide visitors with a long-term opportunity to observe the project in real-time. In essence, this would fulfill the promise of the Urban Research Laboratory by providing interested educators with a testbed to learn about LAN/MAN/WAN technologies in an urban setting which includes schools, libraries and community centers. They would learn about curricular applications, professional development, and political strategies necessary for implementing instructional technology. The goal would be for them to go back to their school districts and begin to implement this technology locally. They would receive support via the technology itself - creating a virtual community of support providers.

b. Who are the near-term beneficiaries?

The beneficiaries of the Urban Research Laboratory would be the students and community served by the Pittsburgh Public Schools - since they would continue the implementation activity already in progress - and other school districts both locally and nationally who would participate in the laboratory and benefit from lessons learned through CK:P.

For Projects that have Transitioned from AAT to NIE, CRLT, or LIS

• What noteworthy challenges have you faced in moving from an R&D emphasis to either implementation or research alone?

CK:P has moved to an implementation phase. The main challenges are political and economic in nature. The district has yet to approve a technology plan due to financial and organizational issues. Without a technology plan, implementation continues to be a political football between traditional power brokers with limited budgets. Although we have a critical mass of users, successes, and experience, and the CK:P model is used throughout the district, the support mechanisms that make Common Knowledge successful are not part of the district's formal administrative structure.

• What are your "lessons learned" that could benefit other projects/programs making a similar shift and how are you sharing those lessons?

There exists a tension between research and production, between teachers and administration, between collaborators. Some of the tension is cultural in nature, some is a matter of priorities and some is a matter of trust. CK:P is a collaborative project that was funded through the universities and yet managed within the school district by teachers. This structure provided the flexibility to conduct research, build culture, develop credibility and create momentum from the bottom up. However, a bottom up approach is not the best method for creating political and structural support within the system. To move to implementation and "institutionalization" one needs to work within the organization's power structure. Projects that take a similar approach need to address political issues in a sensitive way at the soonest possible moment. Given enough time and resources we believe the political barriers can be navigated. This is accomplished through creating successes, sharing the process publically, and involving the appropriate stakeholders when the opportunity presents itself.

CK:P is in the process of detailing its transition from research to implementation. Although this material is online in the form of reports to NSF, we are hoping to compile a more extensive literature on this subject. Both its evaluation team and staff are preparing papers, speaking at conferences and working on larger volumes that address these issues. We hope to use these publications to share our "lessons learned" with the professional community of educators. We also will apply for continued funding to share these lessons through an ongoing Urban Research Laboratory.

For Testbeds

• What role should testbeds play in implementation and scaling-up R&D, and what are your "lessons learned" that could benefit starting testbeds?

Testbeds play a crucial role. They bridge the gap between research and production, allowing for powerful research to occur in real-world, real-time environments. A key lesson is that management of the project should be within the organization that is being acted upon. Another lesson is that technical, support and educational staff should live together in order to create synergy on the project.

• What noteworthy challenges have you faced in implementing a testbed of disparate projects or activities, and what are "lessons learned" that could benefit school-based implementers?

Each year, CK:P brings aboard 3-9 schools. Each school has its own unique needs, curriculum and staff. The same unique needs occur on the classroom level and in the school district as a whole. The lessons learned pertain to taking a metacognitive approach to the process of implementation. We label this "attending to the noise." Data is provided constantly that pertains to the effect of the process on individuals, groups, schools and the district as a whole. One should not write off dissonance. Researchers and implementors should focus on the dissonance, learn from it and allow it to inform their efforts.