Exploring the use of AI-Writing Assistant for Foreign Language Learners: A Mixed-Methods Study in the Saudi EFL Context

I. INTRODUCTION

The rapid development of AI has presented both opportunities and challenges, particularly in the context of chatbots like ChatGPT. This study explores how language learners interact with AI-based writing assistants, particularly in providing feedback to enhance their writing skills. It aims to fill gaps in existing literature and increase understanding of the influence of AI feedback on writing skills. It also seeks to understand how learners use AI-based writing assistants to prompt their input, thereby filling the gap in understanding learners' comprehension of teaching instructions and motivation for learning in AI tools.

As educational landscapes evolve with the integration of artificial intelligence, understanding how learners engage with AI tools becomes increasingly significant. The adoption of AI-based writing assistants like ChatGPT represents more than just technological enhancement-it signifies a paradigm shift in learner autonomy, feedback reception and writing development processes. While AI tools offer instant support, corrections and suggestions, the way learners interpret, evaluate and act on this feedback remains largely underexplored. Moreover, the dialogic nature of tools like ChatGPT allows for dynamic learner-AI interactions, which may influence not only linguistic accuracy but also learners' motivation, confidence and critical thinking. This raises important pedagogical considerations: Are learners equipped to navigate and interpret AI feedback meaningfully? Do they engage with AI as a replacement for teacher feedback or as a supplementary tool? How do these interactions shape their understanding of writing conventions, structure and coherence? By situating AI-assisted learning within broader educational and technological contexts, this study seeks to uncover patterns of learner engagement, potential misconceptions and the pedagogical implications of AI-mediated instruction. The findings are expected to contribute both to AI-enhanced curriculum design and to the theoretical discourse surrounding digital literacies in language education.

This study explores the relationship between learners and AI tools in education and technology. It aims to understand how language learners interact with advanced AI and the types of interactions between learners and ChatGPT. The findings will help language educators determine appropriate teaching strategies and benefit educational tool developers. The study's primary objectives include exploring the nature of interactions between learners and AI, identifying the type of prompts learners use to express themselves, and understanding how learners perceive AI-based. Understanding the following depends on the research questions:

• How do EFL students interact with an AI-based writing assistant?

• What kind of questions do L2 writers ask when using AI-based writing assistant systems?

• What is the perception of the learner of AI- an AI-based writing assistant?

II. LITERATURE REVIEW

2.1 AI Assistants for Second Language and Writing

Nowadays, various language learning opportunities have become possible due to recent advancements in AI technology. For instance, productive skills, such as speaking and writing, require more effort and time to work with teachers and learners to improve their abilities. Artificial intelligence AI-based computer and mobile programs provide interactive and tailored tools for improving writing skills and increasing motivation (Jiang, 2022; Meunier et al., 2022;

Yan, 2023). Moreover, AI in education (AIEd) has three paradigms. First, AI-directed means learners can communicate with AI-powered virtual tutors or assistants. The second paradigm is called AI-supported, a partner collaborating with learners to develop skills like critical thinking and problem-solving. For example, Chang et al. (2021) investigated how an AI-supported writing feedback tool affects EFL learners' writing performance. Through an experimental group that used Grammarly to edit and revise their written texts, whereas the control group did not have access to Grammarly. It resulted in the experimental group outperforming the control group in writing skills, which resented how AI-powered language learning tools in developing EFL learners' writing ability. The last paradigm is the AI-empowered paradigm, when learners are the controllers of their learning and only use AI to supply the resources and tools they need Ouyang and Jiao (2021). However, integrating AI tools in learning and teaching would match the lifestyle of learners outside of the school and reassure teachers about the purpose of any tool and manage their time and effort smartly. For example, Link et al. (2020) presented an ideal hybrid case in which an AI writing evaluation (AWE) tool provided sentence-level feedback and then the teacher concentrated on higher-order writing mistakes.

On the contrary, AI-assisted writings have challenges and limitations that show the normal face of any innovation in our lives. The first dilemma is the lack of human interaction. AI tools enable live conversations with tutors or native speakers, but their self-guided learning approach poses challenges for learners who require personalized and interactive experiences (Khanzode & Sarode, 2020). To illustrate, Marzuki et al. (2023) highlights the positive impact of AI tools on learning, but also highlights the potential for over-reliance on technology when learners become accustomed to solving their difficulties, leading to a lack of creativity. Grammarly's limited error recognition may not accurately identify errors in content and style, and its effect on content and organization is less significant than that of teachers (Ghufron &

Rosyida, 2018). Transparency is crucial for users to comprehend the inner workings and limitations of AI language learning tools, enabling informed decision-making and fostering trust (Ruane et al., 2019). AI's limitations make it a product that constantly evolves, but cannot fully replace the human mind, requiring users to understand its limitations.

2.2 Automated Feedback

Each learner needs to be guided in the learning process to enhance language acquisition. As Wichadee (2013) highlighted, the significant role of feedback would improve learners' proficiency in many aspects of the language, such as grammatical errors, spelling errors, and diction errors. There are many types that educators have to deal with in the learning process, especially indirect corrective feedback, which is given in different ways by the teacher, such as highlighting, underlining, or coding. At the same time, learners will make self-correction and self-reformulation (Bitchener et al., 2005). Nowadays, feedback could be provided by AI writing tools that carry some burden on teachers to be facilitators even though they promote the learners to be actively involved in learning by doing extra work cognitively without shame or fear of making mistakes. AI writing tools have affected learning a second language effectively, like Quillbot, Word Tune, Jenni, ChatGPT, Paper Pal, Copy and Grammarly. Qassemzadeh and Soleimani, (2016) examined the impact of AI writing tools on EFL teachers' writing quality, specifically content and organization. They found that AI tools enhance clarity, promote logical progression and have diverse perspectives on their influence on vocabulary use and growth. The study highlights the positive role of AI in student writing.

The types of feedback: direct, indirect and metalinguistic that guide learners. First, direct feedback positively impacts students by helping them identify their errors and, thus, improve their writing (Qassemzadeh & Soleimani, 2016). For example, the teacher just provides the learner the correct answer, representing straightforward learning. Second, Indirect corrective feedback is when highlighting the errors, in the same manner as when learners are given indications (by means of highlighting, underlining, or coding) by their teacher on errors that they have made (Bitchener et al., 2005). Metalinguistic feedback is a teaching method that encourages learners to independently search for answers and take responsibility for their learning abilities. Errors are identified or labeled and explanations provided using annotations, examples, or both, depending on their nature (Barrot, 2023). The feedback provides real-time, consistent, and correct metalinguistic explanations, which teachers struggle to provide due to time or resource constraints (Barrot). Specifically, the teacher identifies the error and briefly explains the correct usage of the answer that would advance learners' deep thinking for their knowledge.

2.3 Community of Inquiry and Students' Approaches to Learning to Research Design

The CoI framework is a process model for digital instructional settings that includes three key concepts: social, teaching, and cognitive presences (Garrison et al., 2010). It focuses on meaningful learning through community interactions and has been widely utilized to influence the development of quality online education (Garrison & Arbaugh, 2007; Lomicka, 2020; Smidt et al., 2021). By applying the CoI framework, I can evaluate how learners progress through the phases of cognitive presence when engaging with AI feedback, which is crucial for understanding their knowledge absorption and critical thinking development. The framework also allows for an examination of social and teaching presence, which is crucial for understanding learners' interactions with peers and instructors in AI-mediated environments. Analyzing social presence will enable me to identify the types of prompts learners use when interacting with AI tools. This insight will provide understanding of their collaborative learning experiences and how they communicate their learning needs. Therefore, Students' approaches to learning have a strong relationship with students' academic performance and provide a useful framework for understanding the quality of student learning (Ellis & Bliuc, 2019). There are three types of SAL: deep, surface, and organized approaches. SAL concerns the various methods and processes by which students engage in learning activities (Ellis et al., 2008; Thompson, 2013). By combining CoI and SAL, this study situates learner-AI interactions within broader debates on online pedagogy. Both frameworks highlight that the effectiveness of AI integration depends not only on the technology, but also on how learners approach and interpret it. In practice, this means that AI writing assistants can support meaningful learning when accompanied by critical engagement and teacher guidance, but risk promoting superficial learning if adopted passively. The frameworks therefore provide the conceptual foundation for interpreting the study's findings and for framing subsequent recommendations The research examines the impact of engagement strategies on language acquisition and academic success, emphasizing the importance of thoughtful use of online technologies. It also explores the types of questions L2 writers ask when using AI-based writing assistants, providing insight into how learners value tools that reflect their performance.

III. METHOD

The study adopts a mixed design to explore the interaction between L2 learners and an AI-based writing assistant called Type. Learners' interactions (e.g., their prompts, questions, logs) will be stored for quantitative and qualitative analyses. The qualitative method tries to comprehend people's meaning in a specific setting (Creswell & Creswell, 2018). The learners will be asked to use Type to write short essays, and their interaction data will be stored for later analysis. Then, learners will have a survey to examine their experience with the tool. This method would achieve the answers that were out of scope, using a mono-method opinion (Şahin & Ozturk, 2019). Each angle of the mixed method would help to better understand the missing parts, like the qualitative side, which focused on mutual causation, intuition, and deep knowledge in the natural settings (Şahin & Ozturk). This enhanced the comprehension of learners' interaction with AI tools in an educational context. While the quantitative side identified a clear level of learners' background and ability in order to construct the idea of the study well. It involved pre-survey, capturing process highlights and the writing prompts learners used qualitatively and then assessing the interaction quantitatively again, which expanded the perspectives of the research question.

3.1 Participants

This study used a purposive sampling method to select participants based on characteristics relevant to the study's objectives (Andrade, 2021). Twenty-seven Saudi male participants were from the western region of Saudi Arabia, where Arabic was their native language. The participants were in two groups of different classes on the same science track at the university, which required them to have intermediate-level proficiency in English. The range of age was twenty to twenty-three and all were foundation-level learners in the second semester.

The two groups were selected randomly from the science track, whereas the learners in this track had previously had extensive English courses in the foundational year. For instance, the learners' proficiency is regarded as satisfactory when encountering A2, B1 and B2 writing prompts. The setting was a laboratory, and the data collection was done using the laboratory devices at the university, where the participants' teachers were present to facilitate the process. Twenty-seven participants participated in the study. The sample size was appropriate for the study's aims, ensuring that the findings reached were within the sample's parameters.

3.2. Instruments

3.2.1 Type

Two instruments were used to collect the data. First, an AI-based writing system called Type was used to collect data about participants' interactions with AI. Type is an AI-powered document editor that enables learners to produce high-quality content rapidly by creating drafts, modifying material, and recommending what to create next. Specifically, Type integrated Open AI's GPT-3.5 model into a feature-rich document editor to help language learners brainstorm ideas, receive feedback on their writing and grammar use (Type.ai: The all-in-one AI writing assistant, n.d.). The software's user interface was designed for learners at all skill levels (see Figure 1, the user interface). On the Type website, there are prompts used in interaction with AI that are open-ended and quoted from the British Council to match and level up their proficiency. The British Council is an organization that specializes in English language teaching and learning and works with individuals and governments to support learners in building networks in learning English, receiving a high-quality education and gaining internationally recognized qualifications. The prompts were classified into A2, B1 and B2 and have some changes to match the study's objectives, like more explanation to encourage learners to extend with details.

3.2.2 Pre and Post Surveys

Pre and post-surveys were used to assess the participants' interaction with AI and identify pros and cons of their experience with Type. The system usability scale survey (SUS) is a Likert scale that selects the statements with which respondents agree or disagree, with some changes to suit the study (Brooke, 1996). The Likert scale was changed slightly to match the study's needs in clarifying and meeting the objectives' scope in the post survey by changing numbers and the types of questions (Appendix C, the original survey, Appendix D, the post-survey).

The pre-survey contains demographic information and has four questions: age, academic year, and background in utilizing AI assistants, especially in academic areas, such as the primary goals for using AI-based writing tools and the general feeling about using AI tools for learning and writing. Besides, at the end of the pre-survey, two open-ended questions give important insights into user experiences and goals with AI-based writing tools.

The post-survey presented scale questions that allowed users to rank their thoughts about the type and provide useful insights into user experiences. These questions determine whether users want to use Type regularly, find it simple to navigate and value the integration of its numerous capabilities. They also investigate if users feel they require assistance, how quickly they believe others could learn to use it, and their overall trust in utilizing the platform. On the other hand, the open-ended responses can add meaning to scale ratings, helping to explain why participants ranked certain characteristics the way they did. All surveys were written in Arabic and English, allowing the participants to answer freely in their favorite language.

The first phase, the pre-survey, allowed for a more detailed analysis of their success and ensured that all participants feel prepared to use the tools effectively. This enhanced the ability to define learners' input that would align with their motivations for using AI tools in education. Second, participants randomly encountered one of the writing prompts, a short essay and interacted with an AI-assistant writing in Type. The interactions will be analyzed qualitatively and quantitatively to assess the effectiveness of the AI assistance. Finally, the participants will complete the post-survey to evaluate their experience, which could lead to more thoughtful and insightful comments.

3.3 Experimental Manipulations or Interventions

The main data collection involved Type, where all learners' interactions with AI are stored. The participants received an Arabic tutorial on properly using Type. After that, they randomly used one of the writing prompts to write a short essay. The prompts were developed (Appendix B, the developed prompts) to facilitate critical thinking and creativity, encouraging learners to engage deeply with the writing process.

The data analysis was divided into qualitative and quantitative processes. First, the data was analyzed using Saldaña's Framework for qualitative data in NVivo15 and subsequently transferred to Excel for further elaboration. According to Saldaña (2012), 32 coding methods were identified that represent the first or second cycle, with one hybrid approach in between. It described that data should have filters through a specific process to polish it clearly and wellrepresented. Second, Quantitative data from pre- and post-surveys were obtained using Google Forms. Moreover, there were sheets for both surveys to have more explanation and representation in Excel, which would enhance the clarity of the quantitative data.

3.4 Data Collection

The data collection occurred in two steps. The main data collection method was Type. The software stored all learners' interactions with the AI (ChatGPT and Cloud) and their writing. First, participants were given an Arabic tutorial on properly utilizing Type for data collection, ensuring they grasped its features and applications. The data collection took place in the laboratory of the university, where the participants' teachers were present to facilitate the process. After the tutorial, the participants were asked to write randomly about one of three prompts. Their input was then collected into NVivo 15 and subsequently transferred to Excel for further elaboration.

The quantitative data identified the areas in which improvement was needed for Type and AI-based writing assistance in general. Input for the pre- and post-surveys was received and collected in Google Forms. At first, the responses of both surveys were received in Google Forms and there was a function to analyze them in the same place, except for the open-ended questions, which were analyzed in Nvivo 15 and then organized into Excel for explanation.

3.5 Data analysis

The analysis was divided into qualitative and quantitative phases, and Salanda's Framework was used for the qualitative data. 32 initial codes were identified that represent the first or second cycle, with one hybrid approach in between. It described that data should have filters through a specific process to polish the data clearly and well represented. The qualitative data was learners' input in Type and open-ended questions in pre and post surveys. First, the extensive data was in type to cover research questions that started with first cycle to notice any recurring ideas and pattern throughout the data and labeled them by descriptive codes in Nvivo15. The first cycle of coding had seven subcategories: grammatical, elemental, affective, literary and language, exploratory and procedural. Each subcategory has various code kinds, for which the elemental method was utilized with data and made it easy to link with a simple label. At the end of the first cycle, there were 13 codes for the two research questions and then the second cycle refined the codes to be themes. The primary goal during second cycle coding is to develop a sense of categorical, thematic, conceptual and/or theoretical organization from the array of first cycle codes (Saldaña, 2016), which enhanced the codes and developed into themes. Second, the axial and focused coding methods for polishing the data, such as the codes, were nine under four themes for two research questions. The axial coding is beneficial to list each category's characteristics and aspects, find dominant codes, remove redundant codes, and identify the most illustrative codes, which help to group coded data with comparable themes and ideas, whereas the focused coding identified the most significant codes in the data. Finally, all data was organized and presented in Excel in order to provide data visualization via graphs and charts, efficient organizing, a thorough dataset overview, simplicity of usage using a known tool and fast production of summary tables and reports.

The second group of the qualitative data was from surveys, first, the pre and post-surveys had open-ended questions in Google Forms that summarized the responses and then started with descriptive codes because responses are straightforward and did not need to delve in depth analysis. After that it used pattern coding for promoting a unified awareness of what the responses performed together imply. Finally, the data of the two surveys were analyzed in details in Excel due to representing open-ended responses as percentages is a simple and effective technique for identifying the respondents' perspectives on certain subjects. This strategy highlights frequent replies and focuses on the frequency of specific codes, making it easier for audiences to understand the material without requiring complicated theme analysis or complicated explanations. On the other hand, the quantitative data were easy to analyze by Google forms that offered a function of summarizing and reporting the surveys' input, like there was a report for each participant and there were diagrams to represent the total of responses for each question, which eased the process. Moreover, there were sheets for both surveys to have more explanation and representation in Excel so that would enhance the clarity of the quantitative data.

3.6 Ethical Consideration

Prior to data collection, ethical permission was sought, which includes university approval, participant informed consent and data confidentiality. An informed consent form was distributed in Arabic and English, ensuring that participants were aware of the study's objectives. Participants' identities were kept anonymous in accordance with ethical research standards designed to protect their privacy.

IV. RESULTS

The first research question was to discover the nature of the interactions that happened during this process. To answer this question, the data indicated an Interaction Styles theme divided into sub-themes: anticipated interaction and unexpected interaction. It describes how each learner interacted with AI-based writing, especially in the chat box. This theme explains how data can be classified into regular and irregular. While certain data was expected in the chat interactions between learners and the AI, other data revealed unexpected differences in the learners' inputs.

The results showed predictable interactions that learners would have with AI. Twenty learners tended to check on AI for grammar mistakes to improve the writing style and achieve the word count (figure 1). The previous diagram shows how a learner was concerned about the word limit, and thus demanded the AI to add words to his input to meet the requirement.

The revised text contains 188 words. Would you like me to expand on it to reach a specific word count?

Figure 1: A learner interacted with AI in Type's chat box

Learners had assistance from AI during the experiment process in multiple ways, such as generating ideas, proofreading, or reviewing. Learners' prompts were simple and spontaneous in nature and were utilized in their daily lives (figure 2), such as asking a human to help him without realizing the grammar of the prompt. Moreover, this theme identified and shared these interactive prompts with AI to understand what learners need to rely on AI feedback and where they felt to have support through the experiment process like in the beginning or in the middle or at the end, which reflected their academic experiences and how they approached towards challenges, indicating their reliance on AI for assistance.

The data indicates the irregular responses from the interaction of the learners with AI in Type's chat box that kind of the prompts that need to be shared. First, seven learners did not interact with AI-based writing and left their chat empty, which was a surprising action, especially when they encountered A2 and B1 writing prompts. Even though they had input without reaching the required word limit (figure 3), this raised significant questions about their trust in AI or their trust in their ability to success or relating to their motivation to learn English.

In Figure 4, two of them thanked AI for the feedback, which showed a positive attitude toward artificial intelligence tools. Moreover, learners' interaction prompts humanized with AI as it is human beings have their own goals and feelings without realizing it is a machine that was made for a purpose. For example that presented in figure 5 and 6, they treated AI as if they asked humans about its schedule for today in order to catch some ideas. Furthermore, these interaction prompts were repeated in various contexts.

The second research question aimed to identify the types of prompts that learners used to interact with AI tools during the experiment. To answer this question, three main themes covered various aspects to enhance the identification of the kinds of interaction prompts. The themes were the types of prompts by learners, feedback from AI, and error identification prompts.

The data suggests the types of questions that learners utilized in the chat box with AI to complete the required writing prompt. This theme has three sub-themes: style prompts, grammar prompts, and word count prompts (Figure 7).

{"image_source":{"path":"images/ea357a60cfebaed5ba23c1e31fc1d29e42c36ba31ee798ece151e09eced234de.jpg"},"content":"","chart_caption":[{"type":"text","content":"Figure 7: The number of learners that used the specific prompts in Type's chat box"}],"chart_footnote":[]}

44.83% of the learners recurred to improve their vocabulary and tone in any aspect of the language. They always checked if they were using a suitable term or needed a synonym list to choose the best in Figure 8.

In contrast, others focused on simplicity, like in Figure 9, so they could understand what was quoted from AI and be satisfied with their input. Also, others required formality in his input, which was shared with the AI assistant writing in Type in Figure 9, to follow the academic writing style.

Nine learners concentrated on grammar mistakes and interacted with AI to check and rephrase the correct structures for their whole input. On the other hand, eight were different in interacting with AI in Type to correct their grammar, like some of them shared a complete sentence in the chat box that confused them during the process, like in figure 10.

20.69% of learners were concerned about reaching 200 words, which was a condition for each writing prompt they encountered in the experiment. They could not extend their input until they had support from AI to lengthen their answers suitably, which represented the limit as a challenge to their linguistic ability, especially for A2 and B1 writing prompts, as shown in Figures 11 and 12. In contrast, none of the participants completed the condition in the B2 writing prompt.

The data showed that the AI response depended on the learners' interaction prompts in the chat box with the AI that was divided into learners' inputs and AI suggestions. It was titled as AI evaluation feedback for learners' input and AI feedback and suggestions (Figure 13).

{"image_source":{"path":"images/ed76ee24acaf2ef60b1040330ecdddaef9c9f386f2b74f4dd41077e6634e3049.jpg"},"content":"","chart_caption":[{"type":"text","content":"Figure 13: A percentage of the learners scoped for feedback by AI theme in Type"}],"chart_footnote":[]}

The last theme for the second research question stood for the limited ability of AI to deal with some interaction prompts and the insufficient interaction prompts by learners in the chat box. The first subject is errors by AI that covered 36.36% of errors that AI could not transact with all of the learners' interaction prompts because it was beyond AI's ability. As indicated in figure 14, a learner copied the writing prompt and pasted it immediately without any instruction that would distract AI to conduct with it and provide a correct response.

Question three covered the quantitative data collected twice before and after the experiment to capture the learners' opinions on Type. It was presented in two phases to organize their background and reactions before and after the experiment.

The first phase of the collection, the pre-survey, was a demographic form that was indicated in Table 1.

The third question demanded a long answer and described learners' experiences using another recommended AI-based writing tool. The learners' responses in Arabic and English encouraged them to express themselves widely and share their experiences. As shown in Figure 15, one learner provided an example of an AI assistant writing Gemini, which was a good experience for him. In comparison, 34.21% of learners indicated they use AI tools, while 28.95% had never used them. 21.05% of the learners generally liked AI assistant writing, while 10.53% of the participants emphasized their liking for the AI tool because of its ease, and 2.63% of them mentioned how the AI tool was fast.

Figure 15: A learner's experience with another AI tool

The fourth question used a Likert scale approach to focus on the participants' perceptions about using AI assistance (Figure 16). 66% of the learners strongly like AI assistance when writing and learning English, 25% just like it, and 7.4% have a neutral opinion about writing and learning English with AI tools. The percentage reported a positive attitude toward AI tools.

{"image_source":{"path":"images/6ab3e4fdb8e02e6e2459a3ab952bc7eccea62ac756a04ecafaa74d4064f32409.jpg"},"content":"","chart_caption":[{"type":"text","content":"Figure 16: The fourth question in the pre-survey"}],"chart_footnote":[]}

The last question required a short answer that showed the participants' intention to utilize AI-based writing tools. 50% of the learners commented on the same reason for using AI-based writing tools, which were helping them to improve their writing, such as checking grammar, as shown in Figure 17. Meanwhile, 6.67% of learners considered AI to save time and effort. 6.67% of the participants stated using AI to generate ideas for their work. On the other hand, 13.33% of learners refused to mention the reason. Also, 7.41% of learners do not use AI at all.

Post-survey

The System Usability Scale (SUS) is viewed in relation to the Type, its use and its appropriateness. It is derived from SARD: A Human-AI Collaborative Story Generation, a ready-made form modified to match the study's purposes. There were eight scale questions and two open-ended questions. This survey captured the differences of learners after using the Type and drew links with their backgrounds before the experiment, which enhanced their understanding of their input in Type and their motives. As shown in Figure 18, most of the learners strongly agreed to use Type frequently after the experiment.

{"image_source":{"path":"images/add1e6b3a3a922f89b7373600d7b8e5213932ee33a47e029694bbe740786a1d5.jpg"},"content":"","chart_caption":[{"type":"text","content":"Figure 18: percentages of using Type in the post-survey"}],"chart_footnote":[]}

The second question was about the ease of Type, and it appeared that 55.6% of learners strongly agreed with it (Figure 19). While 11.1% of learners were neutral about utilizing Type, 11.1% of them strongly disagreed that the website is difficult, even though there was a tutorial for using it.

{"image_source":{"path":"images/4442c511646085b154de709165dd2badcda82aeac2b47bbe8ecb0c5149ed8287.jpg"},"content":"","chart_caption":[{"type":"text","content":"Figure 19: The second question showed in the post survey learners' responses"}],"chart_footnote":[]}

The third question highlighted Type's functions and whether the learners integrated it well. Type has various functions, but the experiment selected limited settings to achieve the study's objectives, such as chatting with AI to receive feedback for their writing. The AI in Type is Cloud and Chat GPT (Figure 20). For instance, the votes were close to each other: 18.5% of the learners chose neutral and agreed, while 14.5% disagreed.

{"image_source":{"path":"images/4c13f4ee2d98c506714ea6cedd00273ce7776ac87388ed584f422e8c22e39f81.jpg"},"content":"","chart_caption":[{"type":"text","content":"Figure 20: The participants voted for the third question in the post survey"}],"chart_footnote":[]}

The fourth question clarified that learners needed to understand various functions. The researcher and the teacher have already given a tutorial on using Type to enhance them and an illustrative picture in each folder for each learner. The data indicated that 18.5% of the learners agreed and were neutral, as they had assumed previously in the fourth question in the pre-survey that 5.88% disliked Type. Also, in the last question in the pre-survey, 14.81% of learners refused to mention the reason, which may be seen as a negative attitude toward AI tools. In comparison, 7.41% of learners do not use AI in daily life, which is related to their difficulties, as 25.9% of the learners agreed they needed assistance before utilizing Type.

{"image_source":{"path":"images/7f55c83b7a51071734f6907ba880e7524f8b4fdd4220bfe80b8e26db95ce0399.jpg"},"content":"","chart_caption":[{"type":"text","content":"Figure 21: The learners' percentages to have a guide for using Type"}],"chart_footnote":[]}

V. DISCUSSION

5.1 Discussing Qualitative Findings

5.1.1 How do EFL Students Interact with an AI-based Writing Assistant?

The qualitative data answers the first question which explores that each participant has their way of interacting with the AI-assistant writing in the chat box, which is crucial for understanding the nature of interaction that required in the first research question. The theme classified the interaction style of the learners into two different ways that present the learners' comprehension of AI tools' ability and trust.

Interaction style is anticipated interaction and unexpected interaction. First, the group indicates predictable interactions like asking for various improvements to their input. It highlights how the learners accept to rely on the AI tool and share their knowledge with it which shows how this group can be opened for the new material learning that enhances their skills. The qualitative data findings show that 20 out of 27 participants interacted with AI during the experiment on Type, which confirms that learners appeal for feedback in their learning. As the findings suggest, the scope of the learners' interaction is anticipated prompts such as checking grammar, generating ideas, improving style, and word limit. This supports the work of Chang et al. (2021) utilized Grammarly, an AI-powered writing feedback tool, to enhance EFL learners' writing performance.

The experimental group outperformed the control group, highlighting the potential of AI-powered tools in improving writing skills. This captures the nature of the interaction between learners and the AI assistant writing on how the learners utilize AI for feedback in order to enhance their writing skills.

On the other hand, unexpected interaction identifies irregular prompts from the interaction between learners and AI tools. The qualitative data presents seven learners who left the chat box empty, which raises a question about their acceptance and understanding of the effects of the AI's assistance on their learning. For instance, a learner provided two answers presented in separate paragraphs, highlighting the oscillation of making decisions besides their trust in their knowledge and how they ignored the assistant tools to ease and enhance the learning. Moreover, this theme expands the nature of the interaction in each level of proficiency and knowledge with AI assistants during the learning process, such as the suitability of utilizing AI tools in each level. This connects with as Tight's (2017) investigation of Spanish learners, which indicated that while participants frequently used online writing tools, low-level errors were still common in their output. This is strong evidence that the Col's importance aligns well with the CoI framework, which is a model for digital educational contexts that incorporates three essential concepts: social, teaching and cognitive presences (Garrison et al.,

2010). It emphasizes meaningful learning through community interactions and has been widely used to impact the development of quality online education (Garrison & Arbaugh, 2007; Lomicka, 2020; Smidt et al., 2021).

To discuss the second question, the qualitative data is divided into three main themes: the types of prompts by learners, feedback from AI, and error identification prompts, which identify the type of prompt the learners use to express themselves through dealing with AI tools that each theme captures various angles of the types of prompts.

5.1.2 What kind of Questions do L2 Writers ask when using AI-based Writing Assistant Systems?

The first theme is the types of prompts by learners, suggesting the types of questions that learners utilized in the chat box with AI to complete the required writing prompts, such as style prompts, grammar prompts, and word count prompts. First, the frequent prompts that learners seek are the style prompts, which reinforce the learners' comprehension of the best version of their input. Such as asking for suitable writing style, and a synonym list. This was indicated by Utami and Winarni (2023), who conducted a study on three Indonesian students who used AI-assisted writing for academic research. They discovered that AI-assisted language learning tools positively affected students' academic research writing and improved their involvement in tasks. Second, Grammar prompts that were a concern for nine learners in their interaction with AI-assisted writing, like sharing the whole input or a specific sentence to correct. Finally, word limit prompts were widely employed because the writing prompt required a length of 200 to 250 words. The data captures their attempts to reach and expand their input to the requirement, showing their motivation for success in the task. Several studies emphasize motivation with learning, including the work of Marzuki et al. (2023), who stated that the AI tools are regarded as ongoing interactive training that stimulates internal incentive and responsibility for learning. This second theme Feedback by AI addresses the study's second question by showing how the learners' prompts influenced the AI's responses in the chat box. It is divided into AI evaluation feedback and AI suggestions feedback, which focuses on the dark side of AI. First, evaluation feedback frequently filters out the human touch, resulting in a lack of personal detail in responses. The data shows fourteen participants chose AI feedback over their original inputs, which was disappointing because their input would not need that much change. However, they blindly believed this was the best choice. This demonstrates a negative reliance on AI-assisted writing, as learners could fail to identify their exact areas for improvement. In a previous study, Marzuki et al. (2023) highlight the positive impact of AI tools on learning, but also highlight the potential for over-reliance on technology when learners become accustomed to solving their difficulties with AI, leading to negative consequences. Second, AI suggestions feedback, which happened during the experiment, showed that AI convinced learners of the feedback provided in the chat box like list the changes made to their input or the positive advantages of its feedback so learners could understand it and grasp it as the only correct information they needed. This contrasts with educators provide indirect corrective feedback through highlighting, underlining, or coding, while learners make self-correction and self-reformulation, utilizing various methods in the learning process (Bitchener et al., 2005). AI suggestions feedback often offers immediate changes without necessitating learners to engage in self-reflection or error exploration.

The last theme Errors Identification prompts that elaborated on the second question in the study stood for AI's limited ability to deal with some interaction prompts and learners' insufficient interaction prompts in the chat box. The first subject is learners' errors with AI in the chat box in Type, which were covered 63.64% of the time, which shows how the effect of the wrong prompt would confuse AI feedback. For instance, learners shared their input with incorrect prompts, so the AI could not understand what it should be helping with. This strongly relate to Ruane et al. (2019) stated transparency is crucial for users to comprehend the inner workings and limitations of AI language learning tools, enabling informed decision-making and fostering trust. The second subject is errors by AI, which covered 36.36% of errors that AI could not transact with all of the learners' interaction prompts because it was beyond AI's ability. For example, a learner copied the writing prompt and pasted it immediately without any instruction that would distract AI to conduct with it and provide a correct response. Also, the learners' misspelling prompts could affect the AI's reaction, which would repeat the same provided interaction prompt. This relates to the limitation of the AI as mentioned by Ghufron and Rosyida (2018), Grammarly has limitations in error recognition, potentially not accurately identifying errors in content and style, and has less effect on content and organization in writing. To gain deeper insights into participant experience with Type, open-ended question was includes in the post survey that reveals 10.34% encountered technical problems like system glitches which is normal because many participants were utilizing the same account simultaneously.

5.2 Discussing Qualitative Findings

5.2.1 What is the Perception of the Learner of AI-an AI-based Writing Assistant?

This indicates to ascertain how language learners perceive the use of AI-based writing assistants for educational purposes. Pre- and post-surveys were used to assess the participants' interaction with AI and identify pros and cons of their experience with Type.

The first phase, pre-survey, contains demographic information to help define the study scope and highlight the background information of learners' learning with AI assistance. It has three questions: age, academic year, and general feeling about using AI tools for learning and writing. The findings show 66% of the learners strongly like AI assistance when writing and learning English, 25% just like it, which presents a positive attitude toward AI-assisted writing. Ellis and Bliuc (2019) found a high correlation between students' approaches to learning (SAL) and academic success.

In the second phase, the post-survey presented scale questions that allow users to rank their thoughts about the type and provide useful insights into user experiences. The quantitative data indicates that 59.3% of the learners strongly agreed to use Type frequently after the experiment. Also, 55.6% of learners strongly agreed that Type is easy to use, whereas 44.4% of the learners felt confident when using Type. These high percentages demonstrate that learners are eager to incorporate AI-assisted writing into their learning. Also, they tend to show deeper learning and higher interaction with AI-assisted products for educational purposes. It strongly aligns with the CoI framework is a model for digital educational contexts that incorporates three essential concepts: social (SP), teaching (TP), and cognitive (CP) presences (Garrison et al., 2010). It emphasizes meaningful learning through community interactions and has been widely used to shape the development of high-quality online education (Garrison & Arbaugh, 2007; Lomicka, 2020; Smidt et al., 2021) On the other hand, the data indicates that 18.5% of the learners agreed and the same percentage were neutral about the need to understand various functions of Type. Also, 18.5% of the learners confirm that they need background knowledge to use Type, which requires an awareness of the significance of AI-assisted tools in learning, so they cannot be distracted from their own aim. Yılmaz (2020) providing learners with Learning Analytics data in online courses can improve their perspectives of the Community of Inquiry by boosting the self-directed learning abilities of their participation.

VI. CONCLUSION

The results and conclusions of this study indicate several subjects for additional examination, resulting in the following recommendations for future research endeavors. The study recommends that policymakers and educators integrate AI writing assistants in EFL contexts to enhance language learning and critical engagement while maintaining academic integrity. Policymakers should include AI literacy in EFL curricula and invest in teacher training focused on higher-order writing skills. Educators need to differentiate AI use based on student abilities, fostering critical engagement by having learners compare AI feedback with that from peers and teachers. Balancing AI feedback with teacher input is crucial, and assignments should encourage students to document their interactions with AI to promote mindful and sustainable writing strategies. Key research questions focus on EFL students' interactions, inquiries, and perceptions, contributing to a deeper understanding of AI's influence on language learning. The study used a mixed-methods design to explore the interaction between L2 learners and Type, an AI-based writing assistant. Data was collected through pre-and post-surveys, assessing user experiences and feedback. The findings of this study reveal significant insights into how second language learners interact with the AI-based writing assistant, Type. Qualitative data indicates that learners exhibit both anticipated and unexpected interaction styles, highlighting their varying levels of trust and comprehension regarding AI tools. Participants primarily sought feedback on style, grammar, and word count, demonstrating a proactive approach to utilizing AI for writing enhancement. However, challenges such as over-reliance on AI feedback and errors in prompts were also evident. The study reveals a positive perception of AI assistance in language education, with most users finding it user-friendly. However, some learners lack understanding of the tool's functions, emphasizing the need for adequate training and resources. The findings highlight the potential of AI tools in language education and suggest areas for improvement.

ACKNOWLEDGMENTS

As part of the master dissertation, this study has been conducted at a public university in Saudi Arabia.