We scrutinise each aspect of the information we have gathered in order to create meaning. In doing so, the information becomes a source of evidence that informs our review and allows us to make judgments about our practice.

He aha nga hua ka puta mai i nga kōrero nei? | What does our information tell us?

This question leads us to look closely at each piece of information we have gathered. We want to know what it can tell us about our practice in relation to the review focus. We look for:

• issues that come up again and again
• emerging trends or patterns
• anything seemingly insignificant that we need to be careful not to overlook
• one-off or unexpected pieces of information.

There are approaches to analysis that we can select from in making sense of the information we have gathered.

Each of the 3 approaches to making sense of information (descriptive, conceptual, and numerical) allows us to look at information in different ways. We may choose to use several different approaches to make sense of the same piece of information. Alternatively, we can use one approach to look at a range of information.

Descriptive approach

A descriptive approach allows us to make sense of information we have gathered qualitatively by:

• looking for themes or patterns in clusters or groups of ideas and issues in our information
• looking for contradictions where we have pieces of information that seem to be inconsistent with the rest
• using predetermined categories where we have a specific purpose in mind and will have designed our gathering accordingly – this is a targeted approach and may have links to conceptual frameworks or indicators that we have developed as part of our preparation.

A descriptive approach to analysis allows us to draw conclusions from the information in a wide range of ways. We may find exceptions or challenges that we have not previously considered. If we are responsive to such findings, we have the opportunity to explore new ways of looking at what we do and why.

Example

Aratika Educare looked at the descriptive responses they had in their survey results. They used highlighter pens to work through each survey response, using different colours to highlight clusters of comments made about different kinds of communication. As their coding progressed, they began to recognise trends in the information, and some surprises emerged.

Conceptual approach

A conceptual approach allows us to interpret our information in relation to existing key ideas or theories (or statements that we have developed through indicators). There are several conceptual frameworks that we can access. These are usually identified at the beginning of the review process. For example, the Child's Questions (developed by Carr, May, and Podmore, 1998) is a conceptual framework provided by Te Whāriki.

The Focuses of Analysis is another conceptual framework based on the work of Barbara Rogoff (2003). Templates for both are provided in Appendix 2. In the case of Aratika Educare, the reflective questions in Kei Tua o te Pae provided a conceptual framework for analysis of the profiles.

Example

After reading Book 5 of Kei Tua o te Pae, the Aratika Educare review team realised that there were important considerations to be taken into account in their analysis of the profiles. These included making valued learning visible for everyone, fostering diverse pathways, and inviting participation.

Kei Tua o te Pae – Tāhūrangi

As a result of the knowledge we gained, we developed the following criteria for our analysis of the profiles so that we had a shared understanding of what was important:

• opportunities for parents to offer their perspectives in a range of ways (not always in the form of a response to our written learning stories)
• links to home through knowledge derived from both the child and their family
• consistent frequency and quality of entries.

In their analysis, the teachers worked through the profiles – looking for samples where parent perspectives were evident, where links to the home were derived from the child and their family, and where there was consistency in the frequency and quality of records across profiles. They wanted to know how learning was made visible and who contributed.

Numerical approach

A numerical approach allows us to work with quantitative information. It can involve counting how often different episodes occur, comparing sets of figures and interpreting the meaning of clusters of numbers in questionnaires, surveys, or observations in which scales or measures have been used (McMillan and Meade, 1986). A numerical approach to analysis allows us to make comparisons between pieces of information. Combined with descriptive or conceptual approaches, a numerical approach to analysis can provide a strong evidence base to inform our judgments.

Example

The Aratika Educare review team used numerical analysis to count up the frequency of survey responses based on a five-point scale they had used before. Although they found this analysis to be a quick process of adding up how many 1s, 2s, 3s, 4s, and 5s were recorded for each question they had asked, they also found that there had been some confusion.

We recognised that we had not made it clear for parents that a '1' meant 'unsatisfactory' and a '5' meant 'highly satisfactory'. As a result, we discovered that we might get a '1', recording 'unsatisfactory', followed by a very positive comment that seemed to indicate that the interviewed person had interpreted the '1' to mean 'highly satisfactory'. There was also a tendency for a scale like this to polarise people, which may have resulted in a number of 'don't know' responses, of which there were several.

I ako tatou i te aha? | What did we learn?

Making sense of information in the review process provides us with the evidence that will inform our judgments. As a result of the analysis we have completed, we are in a position to identify which aspects of our practice are positively impacting on children's learning and which aspects of our practice need improving. We draw together all of our analysis.

Example

The Aratika Educare review team brought their findings to the next monthly meeting. They wrote the focus for the review on a large piece of paper and stuck it on the wall for everyone to see as they spoke.

How effective are our communication and consultation processes with parents and whānau?

They also recorded the indicators and shared their findings with the meeting.

Teachers are communicating with parents about their children's learning in consistent, authentic ways and at regular intervals.

The teachers concluded that the children's profiles did not always provide consistent information to parents.

Some profiles contained more information and are recorded in more frequently than others. As we have a designated teacher for every child, this meant that some families were receiving richer information about their child and receiving information consistently, while others were not.

Their report stated:

We did find some inconsistencies in the way profiles are completed and how often this is done. We also discovered, through this process, that our profiles offered quite different feedback in terms of content and quantity across teachers. The learning that was being made visible varied from teacher to teacher.

Parents have multiple opportunities to contribute to their child's learning at the centre.

In addition, they described the fact that the profiles usually invited parent feedback rather than “feed-forward” about children's learning.

We recognised that there was some stuff that we took for granted. For example, we realised that parents played a relatively passive role in the profiles. They responded to what teachers had said rather than vice versa. We teachers realised that we probably constrain parent input into the profiles because we offer only little boxes at the end of our learning stories. We don't invite parents to contribute in different ways.

Teachers are available for parents when they need to talk about their child.

The parents and teachers who had analysed the survey results shared the finding that parents had higher expectations for communication than the teachers had recognised.

The survey result showed that there was a clear message from parents about their expectations in relation to communication. Parents expressed their desire to find out in more detail how well their child was doing. Coding also revealed a strong message that parents valued informal chats, other forms of communication, and social opportunities.

One parent said: The review findings showed that the parents of children under 2 years old valued diary records of their children's day, whereas the parents of children over 2 years old did not specifically mention this.

The Director added: The teachers and I were surprised to learn that parents did not always realise that they could talk to staff about their children at any time. We thought that this was a given!

They shared their numerical analysis of the findings: Out of 24 questionnaire responses, 21 rated all aspects of communication very highly. However, we also found some inconsistencies in the results, which called into question the reliability of this information for the review.

Me pehea ta tatou whakatau? | How do we make judgements?

The process of making sense of the information we have gathered provides us with the evidence we need to make sound judgments about our practice.

• Information: The information is what we have identified, gathered and now analysed.
• Evidence: The evidence is the analysed information we use to inform our judgements.

In bringing together our evidence, we now ask:

• What aspects of practice are we doing well?
• What aspects of practice might we need to improve?

Making judgments about what we are doing well and what we might need to improve is often quite challenging. It is a collective process in which the emphasis is on sharing responsibility rather than on apportioning blame. We want to be sure that our judgments are informed by the evidence that has been generated from the review process rather than based on speculation or a hunch.

Our ability to reach effective judgments through making sense of the information is underpinned by the elements of effective review. These are:

• Relationships: the strength of our relationships in challenging and listening to one another.
• Evidence: our ability to use the evidence to inform our practice.
• Vision: the extent to which we have a shared vision for our work.
• Improvement: the commitment we share to improvement.
• Ethical: our ethical awareness that contributes to everyone feeling safe.
• Wisdom: the wisdom we share in making sound judgments.

(See Section 3 for a discussion of these elements.)

The elements of effective review

Sometimes we identify other aspects of our practice that need consideration as a result of our making sense of the information that we have gathered. Alternatively, we discover that more evidence or careful scrutiny of all the information is needed before judgments about our practice can be made.

Example

As their review was initially concerned with collaborative practice, the members of Aratika Educare who were at the meeting were able to agree that there was not a shared understanding about communication between all members of the service. There were some surprises for everyone. For instance, some of the parents said they had not previously thought that they had the option of contributing to profiles in other ways, and they started to talk about using their home videos and infant records.

While there was a reasonable level of satisfaction in general, there was more that could be done to improve communication between parents and teachers about children's learning.

Parents want more opportunities to communicate more specifically about their child's learning. The review indicated that the ways parents wish to communicate varies between the under 2s and the over 2s services. We can't assume that one size fits all.

Aratika Educare recognised that the findings had implications for learning and teaching practice.

They recorded: We have to take more time to think about the learning that we make visible because it gives a message about the learning that we value. We also want to know that every child, regardless of their teacher, is receiving equitable assessment, particularly as we know that assessment is an integral part of learning.