Optimizing Mobile-Edge AI-Generated Everything (AIGX) Services by Prompt Engineering: Fundamental, Framework, and Case Study

PFMs show great potential in scheduling complex networks and systems due to their analytical and reasoning abilities. For instance, Zou et al. [2] employ ChatGPT to schedule the transmission power of senders for saving energy in shared-spectrum communications. By informing ChatGPT the environment (i.e., $n$ senders, $m$ base stations, $r$ rounds, bandwidth $b$ kHz, channel gain $g$, noise $n$ dBm/Hz), game rules (i.e., in each round, each sender chooses its transmission power), and network goal (i.e., the total power consumption does not exceed $p$ W), a qualified transmission power schedule of each sender in any round can be generated automatically.

For efficiency or privacy purposes, the message for transmission can be processed generatively, i.e., using GAI to generate compressed or privacy-preserving contents from raw data. For example, Liu et al. [10] present a generative semantic extraction method for compressing data transmitted over wireless links. Take image transmission as an example. Unlike traditional semantic communications that extract features from raw images, by ControlNet [7], the sender directly generates raw images’ skeletons, which are transmitted to the receiver and decoded by a Stable Diffusion. Due to data size compression, the bandwidth consumption is reduced by half while most semantic features are preserved.

Optimization widely exists in practical system applications as a problem of finding the best solution from all feasible ones. Recently, diffusion models showed great potential in optimization since the denoising process starting from Gaussian noise can be regarded as the optimization process of finding the optimal solution starting from random policies. In light of this, Du et al. [1] introduce a diffusion process to Deep Reinforcement Learning (DRL), which greatly improves the exploration ability and achieves the SOTA performance in many optimization problems, such as incentive mechanism design, wireless resource allocation, and channel coding for wireless communications.

Besides generating human-oriented texts, GAI also supports generating machine-oriented codes in different languages. For instance, Destefanis et al. [11] utilized GPT-3.5 to generate Java codes and proved that AI-generated codes can solve 90.5% of practice coding problems published on CodingBat.com website. Furthermore, code translation, e.g., rewriting MATLAB codes by Python, and code documentation, i.e., generating user manuals for source codes, can also be realized by GAI.

Lesson Learned: The above examples demonstrate that AIGX has the following features:

• •

  Interaction with Environment: Apart from serving human users, in many cases, the “things” generated by AIGX should interact with the environment, such as the AI-generated smart contracts for blockchain systems.

• •

  Sensitivity to Uncertainty: AIGX is less tolerant of unexpected outcomes. In the above example, if the generated code contains loopholes, the digital assets will be subject to a great risk of loss.

• •

  Requirement for Efficiency: AIGX not only concerns the user-level generation quality but also focuses on improving the system-level performance, such as latency, resource efficiency, robustness, etc.

These features put forward urgent requirements for high-quality prompts, only by which the environments can be described clearly, and unexpected generation errors can be alleviated. Moreover, we intend to explore the role of prompt engineering in optimizing not only the user experience but also the efficiency of the AIGX services.

A typical way of prompt engineering is to compose different prompts and search for the one that leads to the best generation quality manually.

• •

  Content Optimization: Firstly, we can refine the prompt content, thereby better associating the downstream tasks with the learned knowledge. For example, ChatGPT prompts can be refined by clarifying the role (i.e., what role that ChatGPT is expected to act as, e.g., a writer), the objective (i.e., what is the purpose of this prompt, e.g., comparing fine-tuning and prompt-based methods), and the outcome (i.e., what is the format of the expected outcome, e.g., a table showing the difference)¹¹1https://the-decoder.com/chatgpt-guide-prompt-strategies/.

• •

  Organization Optimization: The prompting strategy also greatly affects the generation quality. For instance, Fig. 1(d) shows three organizations of textual prompts, namely zero-shot prompting, few-shot prompting, and Chain-of-Thought (CoT) prompting [12]. Firstly, zero-shot prompting means that only a task description is provided. In contrast, few-shot prompting gives PFMs a few demonstrations to showcase the expected output format. Finally, for some deductive tasks, the CoT paradigm can help PFMs split the task into multiple simple subtasks and generate correct results step by step.

Although manual methods are easily accessible, they require the users’ prompting experience and an in-depth understanding of the downstream tasks. Therefore, researchers further present various automatic prompt engineering methods, circumventing the effects caused by human-related factors, e.g., limited knowledge and bias, and achieving stable prompt optimization performance.

• •

  Discrete Prompts: Discrete prompts, such as texts and images, are typically human-interpretable. Take the optimization of the textual prompts as an example. Jiang et al. [jiang-etal-2020-know] presented the prompt paraphrasing. Specifically, each given raw prompt is translated into multiple languages, during which some phrases were replaced by those from a thesaurus. In this way, a series of prompts can be generated, which are then sent to PFMs to achieve the highest generation quality. Moreover, thanks to the evolution of PFMs, the optimized prompts can even be directly generated [4].

• •

  Continuous Prompts: In contrast to discrete prompts, continuous prompts mainly refer to the learnable embedding layer, which pre-processes the inputs before feeding them to PFMs. Consequently, such prompts can be refined by fine-tuning their learnable parameters, called prompt tuning [4].

As shown in Fig. 2, our framework follows a mobile-edge-cloud architecture like NetGPT [14]. The mobile layer accommodates mobile users, who request AIGX services and use the generated “things” to empower various applications, such as Web 3.0 and Metaverse. It is worth noticing that “users” can refer to not only humans who require multimedia content but also machines and networks that require system designs, codes, etc. However, users suffer from limited physical resources. Above the mobile layer, numerous edge servers operate PFMs locally and serve as ASPs in the so-called edge layer. The introduction of an edge layer can bring the following advantages. First, the data transmissions between these two layers are realized by mobile-edge communications. Since ASPs are close to users, low communication latency can be guaranteed. Moreover, users can access nearby ASPs and receive the ubiquitous and customized AIGX services. Finally, the system’s robustness can be enhanced because the introduction of massive ASPs can eliminate single-point failures and also avoid congestion on a single server. The cloud layer supports the lower layers by 1) providing storage services, e.g., the datasets for training PFMs, and 2) operating extremely large PFMs, such as GPT-4 and DALL$\cdot$E 2, with high performance. Accordingly, users are free to call cloud-based ASPs with more powerful PFMs if edge-based ASPs cannot meet their demands. To realize prompt engineering in our framework, Prompt Engineering Service Providers (PESPs) are deployed in the edge and cloud layers. In this case, users can send raw prompts to PESPs and request paid prompt engineering services. Similar to ASPs, users can choose edge- or cloud-based PESPs according to the specific application requirements. Compared with edge-based PESPs, cloud-based PESPs are more powerful since the cloud servers have enough computation and storage resources to support extremely large datasets and machine learning models for training complicated prompt engineering methods. However, since massive users send prompts to one cloud server simultaneously, cloud-based prompt engineering may suffer from long service latency and leakage of user privacy.

In the proposed framework, to request mobile-edge AIGX services and acquire optimal performance, users should perform the following four steps (see Fig. 2).

• •

  Service Configuration: Considering the diversity of PFMs, the ASPs are highly heterogeneous in terms of the AIGX services that they can provide. Therefore, the first step for users is to find suitable ASPs whose PFMs’ capability can meet their specific requirements. The users can conduct multiple rounds of handshakes with target ASPs, confirming the form of the required information (e.g., an image, an article, or an optimization solution), expected service fee, and other configurations. Take AI-generation images as an example. The ASP should run text-to-image models and be able to generate images with different styles.

• •

  Prompt Engineering Configuration: Similar to traditional optimization problems, in which the optimization strategies should be designed according to the specific types of optimization variables, different types of prompts, e.g., textual, visual, or parameterized prompts, also correspond to different prompt engineering methods. Therefore, in this step, the users select suitable PESPs to optimize a given form of prompts. In our example, the selected PESP should be trained on massive image-caption pairs and linguistic corpus, thereby improving the precision of the raw prompt in describing the image.

• •

  Optimization Formulation: Then, the users optimize the AIGX services. Specifically, a series of Key Performance Indicators (KPIs) can be considered to define the objective functions. At the user level, the most fundamental KPI is the Quality of Generation (QoG). According to the specific AIGX tasks, QoG can be defined in different ways. For instance, in our example, QoG can be measured by image quality assessment scores [15]. Higher QoG not only represents higher user satisfaction levels but also reduces the service latency, service fee, and system-level bandwidth consumption caused by re-generations. However, calling PESPs causes extra service fees and bandwidth consumption. Facing this trade-off, the users should define their own QoG and build personalized QoE models. As shown in Fig. 2-Step \scriptsize{3}⃝ , a simple QoE model can take the form of reward - cost, in which the reward is determined by QoG and the cost is related to service latency, service fee, bandwidth consumption, etc. Accordingly, the optimization objective is to maximize the QoE. Moreover, the constraints from users and environments, e.g., the available service fee and resources of users and PESPs, respectively, should be considered.

• •

  AIGX Inference & Iterative Refinement: Finally, the users call AIGX services. Specifically, they can improve the QoE by adjusting the policy for conducting prompt engineering. As shown in Fig. 2-Step \scriptsize{4}⃝ , in each round, the users decide whether to call PESP and measure the resulting QoG. If the value satisfies the users’ requirements, they can enter the next round. Otherwise, they ask ASPs for re-generation. Note that the effectiveness of the selected prompt engineering method in a given task is a posteriori knowledge to the users. Hence, they have to monitor their long-term QoE and choose more powerful PESPs when the current PESP cannot increase QoE anymore.

Our testbed is a MacBook Pro with an 8-Core Intel Core i9 CPU and AMD Radeon Pro 5500M GPU. The text-to-image services of ASPs are supported by Stable Diffusion v2.1 [7].

First, we evaluate the effectiveness of the proposed prompt optimizer in terms of improving the QoG. As shown in Fig. 4(a), the image generated by the raw prompt is blurred and poorly composed, with a NIMA score of 4.8. After prompt optimization, the image quality is significantly increased with modern cooking machines, natural lighting, and higher resolution. Accordingly, the QoG, i.e., the NIMA score of the image, increases to 5.47. Moreover, we perform an ablation study, in which we gradually enrich the prompt content, each time adding one more aspect. Figs. 4(a) and (b) clearly show that, in this case, optimizing the raw prompt in terms of objects & relationships plays an important role in improving QoG. However, as mentioned before, the AIGX inferences contain uncertainty. In other cases, the optimization in other aspects may play a major role. To this end, as shown in Fig. 4(c), we further evaluate many other interior design cases, such as the living room and operation room. We can conclude that, in general, using the proposed GPT-empowered prompt optimizer can improve the image QoG by an average of 11.85% and up to 20.5%.

Then, we explore how prompt engineering optimizes the mobile-edge AIGX services. Fig. 6 shows the QoE of designers for generating 100 images with different prompt engineering rates, i.e., the probability of calling PESPs. We can observe that the higher the rate, the higher the QoG and the overall QoE. Without prompt engineering, the QoG of AI-generated images is only 4.698$\pm$0.471, in which case the designers have to re-generate 284 times to acquire 100 qualified images (see Fig. 6). As shown in Fig. 6, re-generations waste considerable time and network traffic (e.g., 15,402 seconds and 69,310 KB when the prompt engineering rate is 0%), greatly decreasing the QoE. With the increasing frequency of calling PESPs, although the service fee increases, the smaller number of re-generations significantly saves time and bandwidth. In our system, the designers can maximize their QoE by setting prompt engineering rate to 100%. Supported by prompt engineering, 100 qualified images can be generated in 5,692 seconds, with only 25,616 KB bandwidth consumption and 43 times of re-generations.

Dissusion: The above experimental results indicate that QoE can be maximized by always performing prompt engineering. However, such a conclusion ignores the network constraints. For instance, if the capability of PESPs is limited and optimizing each prompt takes a long time, the costs for prompt engineering will increase accordingly. Moreover, as we mentioned in Section II-B, the effectiveness of prompt engineering varies in different tasks. In cases where optimized prompts cannot increase QoG, calling PESPs will degrade the overall QoE due to the extra service fee.