Large language models (LLM) are ubiquitous and used in different domains, for example text generation. How adept these models are at understanding the subtlety and nuance of human-written text is of interest, both from the perspective of generating cohesive content that can be integrated with other text sources, but also from the perspective of content attribution.

Previous research shows that LLMs can understand and analyse human emotions, as well as generate coherent and contextually relevant answers, but they can also suffer from different types of biases and flaws introduced either in the training data, prompting techniques or models themselves. The Appraisal Framework, which captures the subjective presence of writers and their adopted stances in their texts, was used in this research essay. Pairs of texts were analysed and evaluated to determine how generated content differed from both each other and human-written ones regarding stance, sentiment, and engagement. The text generation was first done with Llama3 using prompting only, and then by prompting together with an input sample that it can mimic.

The results indicated that a model-generated text with proper prompting techniques and which had a sample to mimic had few distinguishing differences compared with the input sample from a human author. Naïve techniques used by a prompter for text generation left more markers that could be used to attribute a text. The Llama 3-model seemed to align better with texts that were positive in stance or human-authored, while texts that were neutral or negative in stance experienced shifts of appraisal, as well as amplifications, which made them stand out more as potentially generated.