Modules used: A2, B1

Modules used: A2, B1 – Science Event – 2012

Abstract

This report contains the evaluation of four science communication initiatives during the national Month of Knowledge 2012. Visitors were interviewed using standardized surveys at one of the four selected events. In addition, six professionals involved in the organization of these specific events of the Month of Knowledge were interviewed.

The evaluation shows that all the events are mainly visited by local higher educated male residents aged between 40 and 50. Most of them were accompanied by their family members, that is to say children. Although most visitors find the particular event interesting, the impact on their scientific literacy behaviour is rather weak or non-existent in most events. Most visitors do not want to seek extra information and do not feel enhanced to debate or discuss on scientific issues. Overall, visitors do not want to intervene in decisions on subjects and methodologies used by scientists. They trust scientists although their work needs to be checked/peer reviewed by other scientists.

The professional interviewees showed that they liked to interact with the lay audiences, but most of the time in a one-way mode. None of the interviewees claimed to have learned anything from their lay audience. Both visitors and scientific hosts mainly consider the event as ‘just fun’.

Looking at correlations between gender, age and education, and the questions asked, it was found that women visiting science centre 1 are much more interested in the event than what they have learned about science in school. It was found from science event A that when visitors scored rather high on learning about science they also reported feeling confident in discussions. No significant correlations were found for education.

Introduction

Since the PLACES organization was specifically interested in the evaluation of the Month of Knowledge as an example of public outreach on science and technology in the country, we decided in accordance with PLACES to evaluate four distinct events during this month.

All four events can be considered as SCIP (Science Communication Initiatives and Policies) and were part of the Month of Knowledge programme that was publicly available on the internet. All four of these organizations have a tradition or specialization in doing this typical kind of outreach activity for various target audiences. Except for science centre 1, none of the other three organizations had developed a special programme participating in the Month of Knowledge. None of the cities was a declared science city or a city of scientific culture.

The dimensions of the public (248 surveys) and the level of actors (6 interviews) were studied. Moreover, the cases cover both the level of science centres (science centre 1 and science centre 2) and science events (science event A and science event B).

Overview of organizations and events

Science centre 1 is part of the university of technology of city 1 and is situated in the biggest urban area of the country. Its main task is to interact with the lay audience about the research that is carried out at the university of city 1. It is sponsored by the council of city 1, private companies and various governmental organizations. Most of the exhibits in the museum are hands-on exhibits in which visitors learn to design or decide. They can even do some building or lab experiments. The museum is visited by 13,000 visitors a year, with 30% of the visitors coming from primary and secondary educational organizations.

Science centre 2 is a museum in city 4 which interactively combines art and science . Visitors can do lots of experiments and tests by pushing a button. The various experiments are constantly on tour through the country as well. Therefore, science centre 2 is somewhere between a museum and an event and is based on interaction, science and art. According to its website the museum is open to people from the age of 10 upwards. The funding of science centre 2 is not clear.

Science event B in city 3 is responsible, together with other nearby public observatories , for public communication on astronomy. Throughout the year they organize public meetings and courses on astronomy.

Science event A is situated on the western side of the port of city 2, where a new European location for port activities and industry is being created immediately to the west of the present port and industrial area. When the construction of this area began in 2008, the sea there was 17 metres deep. This area existed on no map, except for the design drawings. This area will soon encompass 1,000 hectares net of industrial ground, located directly on deep water. A visitors’ centre is being built and guided tours provided to discover the new land. The issues explained and discussed are sustainability, the port and economic development, and land-winning technology.

These four cases rather resemble the various events that are presented in the Month of Knowledge and might therefore be seen as a cross section of the events for the Month of Knowledge. In fact, two of them turned out to be rather small events (science centre 2 and science event B) whereas two of them are rather big (science centre 1 and science event A). The latter are also part of big governmental and private organizations, the university of city 1 and the port authority of city 2.

Methods

It was proposed to the PLACES research supervisors that for the present case study the standardized survey formats (module A2) should be used as described in the work package description. The survey was cut short at question 15 since none of the centres or events was declared a city of science or scientific culture. Therefore, it is obvious that people do not know about these qualifications and that if they do know, it is not in connection with the specific event that was evaluated.

The survey questions were translated into the national language to make sure the local audience would understand them properly, with straightforward translations of questions 1 to 14, including questions about demographics and educational level. Also the open answers could easily be answered in the national language (see appendix for the questionnaire translated into the national language). The total number of surveys is 248.

For the interviews we approached six professionals (module B1): two volunteer board members of science event B, the communication manager of science event A, the project engineer at science centre 1, the researcher at science centre 1 and the director of science centre 2.

All events described are part of the national Month of Science. This is a yearly national science event in which many organizations take part.

The researchers visited the events in teams of two. Surveys were handed out to the visitors, and researchers were on hand to explain in case the respondents had any questions. The data were scored in Excel and analysed by using SPSS (see appendices for details). The statistical outcomes are frequencies per question and we especially looked for correlations between demographics such as gender, age and education in order to see if there was any slight difference within the target audience of an event or between events. See appendices for SPSS-reports. The concluding results are described in terms of high, low and trends towards high and low.

Results

Results A2: visitor survey

Overview of appendices

See appendix 1 for translation of questionnaire.
See appendix 2 for demographics per event.
See appendix 3 for frequencies per question/per event.
See appendix 4 for correlation tests .
See appendix 5 for interview transcriptions.

Per event

City 1, Science centre 1:

N =112
65% male / 33% female
Mean age 42.4
The biggest group of visitors are residents from city 1 (36).
Mainly with higher education.

From the surveys we learn:

The large majority were visiting the museum for the first or second time.
Most visitors visit the centre with their families.
They are also accompanied by their families if they visit science centre 1 for the second time.
Visitors tend to find the science centre interesting with a tendency to be more interesting than an art gallery or cultural event.
The way they learn about science and technology is a lot more interesting than at school.
The visit has little or no clear effect on feeling more confident in discussing scientific issues.
Did as many people seek scientific information afterwards as those who didn’t?
If people seek information, the internet is the most popular source.
Visitors tend to believe that science and technology make their lives easier, healthier and more comfortable.
Visitors tend to consider science and technology as something important to know about in their daily lives.
People appreciate the fact that data should be checked first by other scientists before it gets published or is made public.
Visitors agree that scientists disagree.
Visitors find, though not very strongly, that scientists should listen to society when scientific research is concerned.
People disagree with the idea that scientists should adjust their findings to the funders’ interests.

City 3, Science event B

N=25
52% male / 48% female.
Average age 47.
Most people are from city 3 or nearby.
Mainly with higher education.

From the surveys we learn:

Most people were visiting the event for the first time, some of them for the second time in 12 months.
Most people visit the event with their families.
They are also accompanied by their families if they visit the observatory for the second time.
People find this as interesting as an art gallery or cultural event, with a tendency to be more interesting.
The way they learn about science and technology is a lot more interesting than at school.
The visit has little or no clear effect of feeling more confident in discussing scientific issues.
Most people did not answer the question. Of those who answered most did not seek scientific information afterwards or did not know how to.
When they searched for information, the internet was the most popular source.
Visitors tend to believe that science and technology make their lives easier, healthier and more comfortable.
Visitors tend to consider science and technology as important to know about in their daily lives.
People appreciate the fact that data should be checked first by other scientists before it gets published or is made public.
Visitors agree that scientists disagree.
Visitors find, though not very strongly, that scientists should listen to society when scientific research is concerned.
People disagree with the idea that scientists should adjust their findings to their funders’ interests.

City 4, Science centre 2

N=48
50% male / 50% female
Average age 49
Most people live in city 4
Mainly with higher education.

From the surveys we learn:

Most people were visiting the event for the first time, some of them for the second time in 12 months.
Most people visit the event their families.
[scattered answers, not as clear as for the observatory or the science centre].
People find this as interesting as an art gallery or cultural event, with a tendency to be more interesting.
Visitors find learning about science a bit more interesting than at school.
The visit has little or no clear effect of feeling more confident in discussing scientific issues.
Most visitors did not search for extra information.
When they searched for information the internet was the most popular source.
Visitors tend to believe that science and technology make their lives easier, healthier and more comfortable.
Visitors tend to consider science and technology as important in to know about in their daily lives.
People appreciate the fact that data should be checked first by other scientists before it gets published or is made public.
Visitors agree that scientists disagree.
Most visitors are neutral to the idea that scientists should listen more often to society.
People disagree with the idea that scientists should adjust their findings to their funders’ interests.

City 2, Science event A

N=85
65% male/ 35% female
Average age 51.2
All the people came from different cities
Mainly with higher education.

From the surveys we learn:

That, by far, most visitors were visiting the museum for the first or second time.
Most people visit the event with their families.
[scattered answers, not as clear as for the observatory of the science centre].
People find this as interesting as an art gallery or cultural event, with a tendency to be more interesting.
Visitors find learning about science a bit more interesting than at school.
The visit has little or no clear effect of feeling more confident in discussing scientific issues.
Most visitors that visited science event A for a second time did not search for extra information.
When they searched for information, it was on the internet.
Visitors tend to believe that science and technology make their lives easier, healthier and more comfortable.
Visitors tend to consider science and technology as important to know about in their daily lives.
People appreciate the fact that data should be checked first by other scientists before it gets published or is made public.
Visitors do agree that scientists disagree.
Visitors find, though not very strongly, that scientists should listen to society when scientific research is concerned.
People disagree with the idea that scientists should adjust their findings to their funders’ interest.

General results A2

Although the four cases are not comparable due to the variance in amount of visitors (45-112), some general conclusions may be drawn since the events are comparable by their design and aims.

Total number of visitors: 248.
More men than women visited the events (66% male, 34% female).
Average age is 46 (only science centre 1 was visited by children under the age of 10).
Most visitors are residents in the direct vicinity of the event except for science event A, which attracted visitors all over the country.
70% of the visitors had higher education (university college, university).

From the surveys:

Most visitors are not frequent visitors of the event, though all events can be visited more often throughout the year.
Most visitors visit the events with family members.
Visitors who the event for a second time bring their families.
All visitors find the event more interesting than an art gallery or cultural event except for visitors to science centre 2, who find it as interesting as an art gallery or cultural event.
The way they learn about science and technology is more interesting than at school; again, science centre 2 shows a slightly less strong effect regarding this item.
For all visitors the visit has no or no clear effect on feeling more confident in discussing scientific issues with friends or colleagues.
Except for visitors to science centre 1 not many visitors search for extra information after their visit.
When visitors searched for information afterwards, the internet was the most popular source.
Visitors tend to believe that science and technology make their lives easier, healthier and more comfortable.
Visitors tend to consider science and technology as something important to know about in their daily lives.
Visitors appreciate the fact that data should be checked first by scientific peers before it gets published or is otherwise made public.
Visitors agree that scientists disagree.
Except for visitors to science centre 2 visitors find that scientists do not necessarily need to listen to society when choices for scientific research are considered.
Visitors disagree with the idea that scientists should adjust their findings to funders’ interests.

Significant correlations between variables

Are there any significant correlations to be found in the data when gender, age and education are considered? Appendix 4 shows the statistical results. Though not many correlations are statistically sound, we found a few gender correlations and age correlations. All calculated education correlations were not statistically significant.

Gender correlations:

Women tend to bring their families to science centre 2 more than men do.
Women believe that science centre 1 is more interesting than learning science in school.

Age correlations:

At science centre 1, elderly people visit more by themselves than other age groups do.
In city 3 younger people visit the event with their families and older people visit with someone else.
Elderly people visiting science centre 2 tend to go more often with a friend.
At science event A elderly people come with someone else.
A slight indication for science event A that when older people score high on learning about science they find the event more interesting than in school.
For science event A there is a slight indication that older people feel the most confident when discussing scientific issues.
For science centre 1 older people agree most strongly with the fact that before scientific findings are published other scientists have to check them.
For science event A older people agree more with the idea that it is common for scientists to adjust their findings to suit their funders’ interests.

Results B1: Interviews with stakeholders

From the interviews (N=6) we learn overall that:

Question about the interviewee’s involvement in the case:

Though the communication manager is much more professional in her answer than the other professionals, they are all enthusiastic about explaining science or technological development to a lay audience. They also mention, next to the public understanding aim of the event, that they feel socially responsible:

‘I have been doing this for almost five years, they have asked me for this, and every year it is ‘just fun to do’ (Institutional researcher, serious game developer, science centre 1)

‘It is aimed at clarifying techniques and processes used in the project in order to create (public) understanding of the future project’

(Communication Manager, science event A)

Question about what the interviewees got out of it for themselves:

All respondents report that they did not learn anything from their visitors. However, most interviewees say that they got compliments and acknowledgements from their visitors. And one reported that it was much more like a network event for himself

‘Not that I learned anything from the visitors, but they said they appreciated my explanation. And yes, I had some discussions with them. Fun to do, but you are not really learning anything.’ (Voluntary board member, science event B).

‘I met a couple of people who are involved in the same kind of research as me.’ (Professor, science centre 1)

Question about any difficulties talking to the visitors:

All interviewees report that they are talented in presenting and talking to a lay audience due to former experience, jobs or job contexts.

‘As a director of the museum one of his main tasks is translating science as dealt with by scientists to lay people visiting the museum.’ (science event B)

‘It was not difficult at all to talk to a lay audience. He has been involved in public events many times before so he had quite a lot of experience in talking to a lay audience.’ (Professor, science centre 1)

Question about collaboration or interaction during preparation and implementation of the event:

Most interviewees know the organizers or other scientists, politicians, etc. So the interviewees sound like known professionals.

‘Science event B always cooperates with the botanical garden, and at one time it also cooperated with the municipality of city 3. It is a sound cooperation in which we will proceed in the years to come’ (volunteer science event B)

‘I have known the people I work with for years. The same group organizes this event on a yearly basis’ (Institutional researcher, serious game developer, science centre 1)

Question about receiving useful information:

Except for the senior researcher at science centre 1 none of the interviewees reported hearing anything useful from their visitors

‘A couple of serious conversations about the possibilities of using computer games for educational purposes. One visitor actually taught me a complete new background to one of my research topics’. (Senior researcher, science centre 1)

‘No, the public’s involvement has not generated useful knowledge or shown new problems.’ (Communication Manager, science event A)

Question about why the interviewees participated in the event:

The interviewees’ answers range from ‘pure personal hobby’, via ‘just for fun’ to ‘contribute to the urgency to explain to a lay audience’.

‘Reason for me to team up is pure hobby satisfaction’ (volunteer, science event B)

‘On the one hand, the activities contribute to the perception of port expansion among the public. On the other hand, the activities contribute to the public’s appreciation of technology, both among young people and older technology lovers’ (Communication Manager, science event A)

‘It is just a nice day both for the audience as well as for me’ (Institutional researcher, serious game developer, science centre 1)

General results B1

Although six interviewees is a rather low number to draw general conclusions from, some general observations may be described since the events are comparable in their design and aims.

The interviewees like to talk about their interests and love explaining them to a lay audience. However, there is not that much interaction, let alone a dialogue on scientific issues or discussion on other points of view. In that sense, the events researched are rather one-way events, from researcher to lay audience.
Most of the time they think they did a proper job of telling the audience what it’s all about. Overall, they do have some experience in communicating with a lay audience.
Although there are professional differences between the interviewees, in the end their attitudes towards an audience and their actual reported behaviour doesn’t differ that much.

Conclusions

If we take the general conclusions A2, general conclusion B1 and correlation studies together, some final four-case-study conclusions may be drawn.

As we look at the data, all four events show a quite similar view on how a mostly elderly university-educated lay audience reports these kinds of event. Most of the time they find it interesting and they tend to find this important for their daily lives. But the impact on, for example, feeling more confident about science and technology when discussed or explained is much lower or even non-existent. Also, except for the visitors to science centre 1, most visitors not search for extra information. And if they do, the internet is the most popular source.

There is not that much interaction between the visitor and the scientific host, as reported by the latter. However, scientists feel acknowledged by the visitors and in return, as has been said before, the visitors are positive. Moreover, the attitude towards the audience and the enthusiasm with which scientists, volunteers and communication managers cooperate do not differ much.

Overall, visitors have a positive idea about science and scientists, and find that scientists themselves should decide what to research and how to use the outcome of that particular research. Except for city 3, though not significantly, visitors find that scientists should not listen that much to society.

Looking further into these correlations, age is the strongest indicator for slight differences in finding scientists trustworthy and feeling confident in discussions on scientific topics.

Overall, we can conclude that the events studied are most appreciated by the elderly, male, university-educated audience; that most visitors and scientific hosts consider the event as ‘just fun’; and that visitors do not show much interest or information search behaviour to learn more about science and technological developments and their impact on society.

Recommendations

The results mostly resemble the outcomes from other impact research such as the Eurobarometer on Science and Technology (2010). However, the audience of this country does not report the urgency of public consultancy for new science. This might have to do with a relatively higher trust in authority/experts in the country. However, recent discussions and books suggest that this trust is declining nowadays.

Below are some recommendations on science communication and methodology.

Science communication

Since age and gender are the only variables with any significant correlation, these kinds of event might make use of this in defining their visitors’ profile more precisely.

The fact that the age is rather high may be because mostly the parents fill in the questionnaire. The questionnaire should therefore be made applicable/readable for children as well.

The four events evaluated in this study differ only slightly in their outcomes. This might indicate that the events are much alike. However, this might also indicate that the surveys as such are not precise enough regarding the specific events.

The illusion of dialogue between scientists and a lay audience during these kinds of event becomes clear from the interviews. All respondents report that they have not learned much from the audience. Most of the time they explain and educate.

If one of the focal impact issues of public communication on science and technology is to empower people in thinking about science and discussing it amongst their relatives and colleagues, it might be more useful to organize events that do not focus on information but on empowerment of using scientific and technological information for one’s own benefit. That means that exhibits, events, etc. should be much more personalized with self-learning loops up to second-order learning.

As for the situation of this country, we get the idea from the surveys and visiting the various events that people do not know about the Month of Knowledge. They just visit some event in their direct vicinity, not being aware of the organizational context.

Methodology

As stated in the introduction of the PLACES work package (reference 2 p.10-12), the results can be interpreted when one considers: external circumstances, the role of others, the summative effects, comparison with other activities, SCIP are very diverse in nature and are carried out by a vast array of agents, difficult access to relevant literature, unequal distribution of published research, evaluator-evaluee relationship, disciplines, methods and tools.

The problem is indeed that all variables change all the time. That makes comparison and correlations rather difficult at least. Longitudinal instead of cross-sectional might generate insights in how things are changing in time, whilst the target audience (a fixed group of individuals) does not change over time or is at least controllable. Moreover, individual differences can be detected between visiting an art gallery or a science museum. Comparisons can be made with longitudinal studies in medical sciences, psychology and education on how individuals learn. Even the up 56-documentary is illustrative from that point of view. In this most interesting documentary, individuals are interviewed at specific points in their life up to now. Much is learned about the issues that changed their lives, as well as the individual characteristics that did not change a bit. A quick and dirty Google search showed a tradition of longitudinal research in science education (reference 4) but this tradition is lacking in science communication. Only one study by Tydén (reference 5) was obtained that has to do with science communication.

Last but not least, a 0-measurement is needed to report about any difference in the impact concerning these events. A longitudinal research design may provide/ask for such a 0-measurement.

References

We did not perform any literature study since surveys and interviews were standardized. The few references used are bound to the discussion.

1. Eurobarometer on Science and Technology (2010). Obtained from website European Union, December 2012.

2. De Semir et al. (2012) The PLACES toolkit for the impact assessment of science communication initiatives and policies. Barcelona: Universitat Pompeu Fabra.

3. Dijstelbloem, H, and Hagendijk, R. (2011). The disputed authority of science. Amsterdam: Van Gennep. (Uncertainty is king. The authority of science discussed).

4. Gibson, H.L. and Chase, C. (2002). Longitudinal impact of an inquiry-based science program on middle school students’ attitudes towards science. Science Education, 86(5), 693-705.

5. Tydén, T. (1996). The contribution of longitudinal studies for understanding science communication and research utilization. Science