Guidelines

Results of the Portuguese materials piloting

Report on the results of the questionnaire applied to teachers regarding the use of www.languagesbysongs.eu for teaching Portuguese

 

In its testing phase, the materials for Portuguese language have been tested by 13 teachers that used the material in classes of learners. The testing of the didactic units took place into seven countries: China, France, India, Portugal, Romania, Serbia and Spain. The number of students in a class and their level of language knowledge varies from case to case. In this way, the suitability and usability of the material was tested in several contexts, determining to what extent and under which circumstances the material can be used for the above mentioned groups.
The questionnaire enclosed closed and open questions, setting some quantifiable indicators, but at the same time leaving enough space for own opinions and suggestions for improvement. Since the product was in its testing phase, the opinions of the testers and their view on what can be changed, highlighting the weak and the strong points of the material have played a crucial role. Therefore, special attention was given to this.

Overall results

A. Quantitative results

1. General information on language taught and students

As mentioned before, the questionnaire was filled in by 13 teachers. Most of the teachers (77%) were female.  

1.1.  Type of language appraisal

All the teachers involved in this testing phase, taught Portuguese as Foreign Language.  

1.2. Age of the people targeted

The total number of students was 380, a number high enough to allow the extraction of representative results, especially considering the distribution of students by different age groups. In fact, the target largest group of the experimental process comprised higher education students (18-25), followed by the group of adults (> 25), and smaller numbers of young people (15-18), as shown in the diagram below. The overall duration of the piloting phase of the didactic materials covered the period of three months from 01/02/2013 until 30/04/2013.

Therefore, it can be stated that the testing of the material has been applied by teachers to a variety of age groups, covering a wide area of prospective learners, and ensuring testing on how the produced materials can be applied to the different groups and to what extent they are suitable for all these groups.  

1.3. Total number of learners per class and the proficiency level of students

The graphs presented below show respectively the percentage relative to the total number of students per class, which was mostly (54%) between 11 and 20 students, and the proficiency level of students, mostly A2.

The considerable variation in the size of the groups holds in the advantage that the learning material can be tested also with regard to the suitability of use in small and in large groups, making it possible to identify if the learning effect is the same, and therefore to reveal if the method delivers equally effective results regardless of the group size.

The language knowledge of the groups of learners on which the learning material was tested was mostly situated between A2 and B1 and, in some cases, also at level A1 or B2. Therefore, it can be stated that the testing was applied to students having various language knowledge levels, enabling to determine if the product is suitable for all these, and to define if any improvements or changes need to be taken in order for the learning materials to be used by as many learners as possible.    

2. Feedback on the language learning material

The language material was provided to the teachers, but freedom was given to them to choose which and how many units they will use in class, allowing this way to choose those materials that they regarded to suit best the interests and the knowledge level of the students.  

2.1. Number of used units

Taking into account the above mentioned aspects, we asked our testers to provide us information regarding on the number of units they have used and to specify which units exactly were drawn up for their lessons. The number of songs tested by each teacher ranged from 1 to 5, with most experiencing several songs, as the following diagram shows.  

     

With regard to the opinion of the teachers on the appropriateness and degree of motivation of the songs used, the results were very positive.

62% of testers consider the content of the units suitable to the proficiency level considered while 38% consider it partially suitable. Furthermore 93% of the testers consider the chosen songs motivating to learners. All the teachers consider the content of the chosen songs relevant from a cultural perspective, in order to raise the interest of the learner for learning the language. 77% consider the vocabulary provided adequate to access the interpretation of the meaning of the song.

Finally, 92% say that the component dedicated to structural competence is sufficient to understand the content of the song and the exercises are appropriate to achieve the objectives proposed for the language teaching unit.

2.2. Improvement of the learner's knowledge through the material

In a further step testers were asked to assess whether the level of language knowledge was improved after the use of these materials. They were asked to analyse the improvement from following points of view: listening, speaking, writing and reading skills, structural and intercultural competence.

In general, testers observed an improvement in all those 6 aspects, confirming the usability and the effectives of the training material.

The following table shows in which area the testers regarded the material to be useful. While all teachers remarked improvements in reading skills, not all of them considered that this method contributed to a visible improvement of the other skills at stake, even if for most of them it was useful at least partially. Only writing and speaking skills received two negative answers regarding the effects of the units taught on the development of students on this skills.

To sum up, at least 9 from the 13 testers regarded the material as useful in developing all six above-mentioned language skills.      

2.3. Suitability of the teaching material to fulfill its aim  

In the next part of the questionnaire the testers were asked to express their experience by selecting the several relevant statements about the teaching with this methodology, through closed questions. Below you will find the main data, by which the FADO Portuguese partner was able to evaluate the levels at which the proposed methodology and materials were more and less efficient both for teachers and their awareness regarding students’ response to work done in class. The most highlighted statements are related to the empathy of students to this methodology (26%), the feeling of learners’ integrated development experienced by teachers (26%), and the efficiency of the project in the development of learners’ communicative skills (22%).  

2.4. Groups for whom the learning material is considered to be most appropriate  

The results of the diagram below are related to the following question asked to the teachers:

Identify the target audience of the teaching units proposed by FADO (tick the statements that apply).  

42% percent of the teachers considered the project suitable to all kind of learners, which proves its flexibility and wideness of application. There were also some specific groups mentioned by the teachers as appropriate target groups for this methodology, namely teenagers (23%), and not motivated learners (15%), because the use of songs and karaoke makes students more active and willing to learn with comparison to more conventional learning methodologies.  

The teachers selected a set of assertions as the most significant in relation to their personal and professional point of view with regard to the work performed. All were very clear on saying that this work did not represent an increase of the problems faced at this work (0%), and only 3% thought it did not fit their initial expectations. In fact, a much larger percentage, 40%, of the teachers considered that their experience in this testing work was positive, in such a way that they are available to participate in another project of this kind, with 36% highlighting the benefits of such work to their personal development.      

 

B. Qualitative results  

In order for the results of the testing to be more precise and to allow the material developers to assess the strengths and weaknesses of the learning material, a set of open questions were addressed. These offered the teachers the opportunity to express their thoughts on several aspects of the product.

Below you will find the addressed questions and the answers of the teachers.  

Understanding of tasks and aims  

In order to measure to what extent the indications of the material were clear enough for the teachers to use the material without any problem, following question was addressed:

“Tell whether the objectives to be achieved through the task of applying the methodology and materials proposed are clearly defined and explained before starting your task of experimentation. In case you needed additional information or support, explain the issues that need clarification.”

Except one all the investigators responded positively to the question:

In conclusion, the indications and the objectives have been regarded as very clear by the testers, and no further clarifications were needed. Therefore, regarding this aspect, the material shows a high quality.  

Satisfaction of the teachers with the training material  

A set of questions was designed to measure the satisfaction of the teachers with the material. For this, two questions were asked.

In a first step, teachers were asked if they would use the material in the future. 12 of them answered that they would surely and one replied maybe

In a second step, they were asked if they would recommend the approach and the use of the teaching materials to their colleagues. All 13 persons answered yes.  

Recommendations  

The experimenters produced several comments and made ​​several recommendations. Below, we transcribe up some of these comments and recommendations.