Exploring the Depths of iOS 18 and iPadOS 18: Apple Intelligence and Beyond

Apple's annual software updates for its mobile devices often bring a mix of subtle refinements and headline-grabbing features. While company executives are known for their enthusiastic descriptions, proclaiming each new release as the "best" or "biggest" yet, the reality doesn't always match the hyperbole. However, with iOS 18 and iPadOS 18, Apple might just be living up to some of that hype. These latest iterations of the operating systems deliver the expected quality-of-life improvements but are truly defined by the introduction of Apple Intelligence, the company's ambitious entry into the world of artificial intelligence.

The integration of AI has been a dominant trend across the tech industry in recent years, with companies like Google and Samsung rapidly incorporating AI capabilities into their products. Apple Intelligence represents Apple's comprehensive response, weaving AI-powered tasks throughout the operating system. Even the long-standing digital assistant, Siri, receives a significant upgrade, although some of the most anticipated features are still slated for future releases. This deep dive explores the most impactful new features in iOS 18 and iPadOS 18, explains the nuances of Apple Intelligence, and provides guidance on how to access these updates.

What Is Apple Intelligence?

At the heart of the iOS 18 and iPadOS 18 updates is Apple Intelligence. This is the umbrella term Apple uses to describe its suite of generative AI capabilities integrated across its ecosystem. It's powered by Apple's own large language models, which, according to Axios' Ina Fried, were partly trained using data from the public web. These models are designed to run efficiently on Apple silicon, the company's custom-designed chips.

Apple Intelligence leverages a hybrid processing approach. Many AI tasks are handled directly on the device, taking advantage of the neural engines within Apple's chips for speed and privacy. However, for more complex requests that require greater computational power, tasks are securely sent to Private Cloud Compute. Apple emphasizes that this cloud infrastructure, also running on Apple silicon, is designed with user privacy paramount. Data sent to these secure data centers is used solely to fulfill the user's specific query and is never stored or used for training purposes, a key differentiator Apple is highlighting in the AI landscape.

Apple Intelligence Compatible Devices

It's crucial to note that despite iOS 18 and iPadOS 18 being compatible with a wide range of devices, Apple Intelligence is more exclusive. Its demanding processing requirements mean it is only supported on devices equipped with specific Apple silicon processors. This includes:

• iPhone 15 Pro and iPhone 15 Pro Max (A17 Pro)
• iPhone 16, iPhone 16 Plus (A18)
• iPhone 16 Pro, iPhone 16 Pro Max (A18 Pro)
• iPhone 16e (A18)
• iPad Mini (A17 Pro)
• iPad Air (M1, M2, M3)
• iPad Pro (M1, M2, M4)
• MacBook Air (M1 and newer)
• MacBook Pro (M1 and newer)
• iMac (M1 and newer)
• Mac Mini (M1 and newer)
• Mac Studio (M1 Max and newer)
• Mac Pro (M2 Ultra)

This limitation means that while many users can enjoy the general improvements of iOS 18 and iPadOS 18, the full suite of Apple Intelligence features is reserved for users with newer, more powerful hardware. This strategy aligns with Apple's history of using new software features to drive adoption of its latest devices, particularly the iPhone 15 Pro and iPhone 16 models.

Apple categorizes the core capabilities of Apple Intelligence into three main areas: Language, Images, and Action.

Apple Intelligence: Language

Generative AI is integrated into writing tools across the system. This includes features like Rewrite, which can help users adjust the tone of their text for different contexts, such as making an email sound more professional or a message more casual. Proofread assists in correcting grammar, spelling, and punctuation errors. Users can also quickly summarize selected text with a simple tap, streamlining information consumption.

These language-based enhancements extend to how users manage information and communications. Priority notifications leverage AI to surface the most important alerts at the top of the notification list, providing concise summaries for quick understanding. This feature, initially paused for news apps due to accuracy issues, now presents summarized notifications as italicized text. Similarly, the Mail app introduces Priority messages, automatically elevating crucial emails to the top of the inbox. Users can also get summaries of lengthy email threads and utilize a Smart Reply function to generate quick responses, including answers to multiple questions within an email, reminiscent of features seen in Gmail.

For productivity and information capture, the Notes, Voice Memos, and Phone apps gain the ability to record audio and generate transcripts. Apple Intelligence can then create concise summaries of these transcripts, making it easier to review meetings, lectures, or phone calls. Apple has stated that in regions requiring two-party consent for recordings, the system will notify participants when a call is being recorded.

Finally, a new Focus mode called Reduce Interruptions utilizes Apple Intelligence's understanding of personal context. This mode intelligently filters incoming notifications, allowing through only those deemed relevant to the user's current activity or important contacts, such as a text from a babysitter while you're out.

Apple Intelligence: Images

The image-related features within Apple Intelligence primarily focus on generative AI for creating new visuals and enhancing existing ones. Image Playground is a central experience for this, integrated into various apps and also available as a stand-alone application. Users can generate images based on text descriptions, explore suggested concepts, or even create visuals inspired by people from their Photos library. The tool offers control over the image style, including Illustration, Animation, and the recently added Sketch style, which provides a hand-drawn aesthetic. The generated images can be tweaked to match the context of the app being used.

Perhaps one of the most talked-about features is Genmoji. This allows users to create custom emojis directly from the keyboard using text prompts. You can describe the emoji you want, and Apple Intelligence will generate it. You can even use a photo of a person from your library as inspiration, providing a description to guide the AI in creating a personalized emoji based on their appearance. The generated Genmoji can be refined by adjusting the text prompt.

Image Wand is another generative image feature, specifically designed for the Notes app. Using an Apple Pencil or a finger, users can circle a sketch to have Apple Intelligence refine it into a more polished image. Alternatively, circling an empty space can prompt the AI to generate an image based on the surrounding text in the note, offering a simple way to add visual elements to handwritten or typed notes.

The Photos app also benefits from several AI-powered features. Memory Movies allows users to create personalized video montages by simply describing the desired theme or content, such as "All the desserts I ate during the summer last year, set to a pop song." Apple Intelligence will then select relevant photos and videos, set them to music, and automatically generate a shareable movie. Searching for specific photos becomes easier as well; users can describe what they are looking for, and the AI can find it, even identifying specific moments within video clips, similar to Google's Ask Photos feature. Additionally, the new Clean Up tool in the photo editor enables users to easily remove unwanted objects from the background of their pictures, a function comparable to Magic Eraser on Pixel phones and Samsung's Object Eraser.

Apple Intelligence: Action

The third pillar of Apple Intelligence focuses on enabling the system to perform actions for the user, primarily through a significantly enhanced Siri. The assistant sports a new visual indicator – the edges of the screen glow when activated – and finally gains the ability to handle typed requests in addition to voice commands, a feature that has been available on Google Assistant for years.

The goal is for users to interact with Siri more naturally, with the assistant demonstrating a much-improved understanding of context. For instance, a user could say, "Play the podcast my wife sent me the other day," and Siri should be able to locate and play the specific podcast based on recent messages. However, this advanced personal context capability has been delayed by Apple, citing that the feature is "taking longer than we thought."

Siri can also provide instructions on how to perform certain functions on the device, offering built-in guidance. If a user makes a mistake mid-sentence while speaking to Siri, the assistant is designed to understand and correct for the slipup, and it retains the context of previous queries within a conversation, eliminating the need for repetition. Crucially, Siri can now understand what is displayed on the screen when activated, allowing users to perform actions based on the current content. For example, if a text message contains an address, activating Siri allows the user to ask it to add that address to the sender's contact card. This contextual understanding is expected to work across both Apple's own apps and compatible third-party applications.

ChatGPT Integration and Visual Intelligence

Recognizing that some tasks may require capabilities beyond its on-device and Private Cloud Compute models, Apple has partnered with OpenAI to integrate ChatGPT into Siri. Apple states it is using the GPT-4o model. Users have control over when their requests are sent to ChatGPT, ensuring privacy preferences are respected. This integration powers features like asking Siri questions about the content of photos or documents, including summarizing large PDFs. The Compose feature, which allows generating original images and text from queries, also leverages ChatGPT. These GPT-powered features are available for free without requiring an OpenAI account, though subscribers can link their accounts for access to paid features.

Apple also introduces Visual Intelligence, its answer to features like Google Lens. This capability allows the device to understand and provide information about what the user is seeing. On iPhone 16 models, the new Camera Control button can be pressed and held to activate the camera viewfinder not for taking a photo, but to "Ask" ChatGPT about the subject. This can be used to identify actors on a movie poster, learn about a type of flower, or get more information about a landmark. Visual Intelligence can also extract actionable information, such as scanning a flyer to add an event to the Calendar. With iOS 18.4, Visual Intelligence was extended to the iPhone 15 Pro and 15 Pro Max, accessible via the Action Button or Control Center, similar to the iPhone 16e.

Top iOS 18 Features

Beyond the expansive Apple Intelligence suite, iOS 18 brings a host of other significant updates and refinements that enhance the user experience, privacy, and functionality. While Apple's official preview lists numerous smaller changes, the features below represent some of the most impactful additions.

Get Rid of the App Grid

After nearly two decades of a rigid app grid layout, iOS 18 finally grants users the freedom to arrange app icons and widgets anywhere on the home screen, a level of customization long available on Android. This allows for more personalized layouts, letting users place icons and widgets where they are most convenient or aesthetically pleasing. Further personalization options include tuning app icons to a specific color to complement the wallpaper or converting them to dark mode. Users can also adjust the size of app icons and widgets.

More Control in the Control Center

The Control Center, accessed by swiping down from the top-right edge of the screen, is now significantly more customizable. It features multiple pages or tabs that users can swipe through, including a main page for favorites, dedicated controls for media playback, and smart home device management. Users have granular control over the size and layout of the controls within the Control Center. This customization extends to the lock screen, where the two shortcut icons at the bottom can now be swapped out for more useful functions. Apple is also opening this up to third-party developers, allowing for new controls from other apps to be added.

Lock and Hide Apps

A significant privacy enhancement in iOS 18 is the ability to lock or completely hide apps. Users can choose to require Face ID, Touch ID, or their passcode to open specific apps, preventing unauthorized access. For even greater privacy, apps can be hidden entirely, moved to a dedicated, locked folder in the App Library. When an app is locked or hidden, its information is also concealed from system-wide searches, notifications, and other areas, ensuring sensitive data remains private. This feature mirrors similar privacy options like Google's Private Space in Android 15.

Redesigned Photos App

The Photos app receives a major overhaul in iOS 18, featuring a redesigned interface focused on better organization and discovery. The top of the app displays a familiar photo grid, while the area below offers swipeable collections based on themes like Recent Days, Trips, and People & Pets. This new structure is complemented by the Apple Intelligence features mentioned earlier, such as the Clean Up tool for photo editing and enhanced search capabilities that allow users to find images by describing their content.

Additional organizational tools have been added to give users more control over their libraries. These include the ability to filter collections by date (oldest to newest), sort albums by Date Modified, disable specific collections like "Recently Viewed" and "Recently Shared," reorder items within Utilities and Media types collections, and show or hide items in the Library View that are not part of a specific album or were synced from a computer.

RCS and Satellite Messaging

One of the most anticipated, albeit quietly announced, features is the addition of RCS (Rich Communication Services) support. RCS is the modern texting standard that offers features similar to instant messaging apps, such as typing indicators, read receipts, and improved media sharing quality. While Android phones have supported RCS for years, communication between Android and iPhones has relied on the older SMS/MMS standard, leading to a subpar experience and the infamous "green bubble" divide. With iOS 18, Apple is finally adopting RCS, promising "richer media and deliver and read receipts for those who don't use iMessage." While texts from Android users will still appear in green bubbles, the underlying communication will be significantly improved, potentially ending the broken texting experience that has frustrated many users.

For users without Wi-Fi or cellular service, iOS 18 extends communication capabilities with satellite messaging support on iPhone 13 models and newer. This feature allows users to send text messages via satellite when off the grid, similar to the existing Emergency SOS via Satellite feature. Apple states that iMessages sent via satellite are end-to-end encrypted.

The Messages app itself also receives several new features. Users can add animated effects to any letter, word, or phrase, with suggestions appearing as they type. The Tapback feature, which allows reacting to messages with emojis, now supports any emoji or sticker. A highly practical addition is the ability to schedule texts to be sent at a specific time, a standard feature in many other messaging platforms.

A Password App

Apple is introducing a dedicated Passwords app, building upon the iCloud Keychain functionality. This new app aims to be the central hub for managing passwords, passkeys, Wi-Fi passwords, and verification codes across all of a user's devices. It supports end-to-end encryption for security and will be available not only on iPhone, iPad, and Mac but also on Windows via the iCloud for Windows app. With AutoFill enabled, the app can automatically populate login credentials, potentially reducing the need for third-party password managers for many users.

iPhone Mirroring

While technically a feature of macOS Sequoia, iPhone Mirroring is a significant cross-device capability enabled by iOS 18. It allows users to view and interact with their iPhone's screen directly on their Mac. The iPhone Mirroring app appears in the macOS Dock, providing full access to the iPhone's interface using the Mac's keyboard and trackpad. Audio from the iPhone can play through the Mac's speakers. The feature supports drag and drop, making it easy to transfer files between the mirrored iPhone screen and the Mac. Notifications from the iPhone also appear on the Mac; clicking one will launch the mirrored iPhone interface, allowing the user to respond without picking up their phone. During mirroring, the iPhone remains locked, ensuring privacy.

Top iPadOS 18 Features

iPadOS 18 shares many of the core features introduced in iOS 18, including Apple Intelligence, the redesigned Photos app, the new Passwords app, and enhancements to Messages, Mail, and other system-wide functions. However, iPadOS 18 also introduces features specifically tailored to the iPad's larger screen and Apple Pencil capabilities.

Smart Script

Leveraging the Apple Pencil, iPadOS 18 introduces Smart Script in the Notes app. This feature uses AI to refine handwritten notes, making them more legible while preserving the user's personal handwriting style. Users can scribble quickly, and Smart Script will automatically smooth and neaten the text. A particularly neat trick is the ability to paste typed text into a handwritten note and have it automatically converted to match the user's handwriting style. Users can also easily erase text by simply scratching it out with the Apple Pencil.

The Calculator App

After years of absence, a native Calculator app finally arrives on the iPad with iPadOS 18. However, this isn't just a basic calculator. It includes a powerful feature called Math Notes, which integrates with the Apple Pencil. Users can write mathematical equations directly in the Notes app, and the Calculator app will automatically compute and solve them as soon as an equals sign is drawn. Math Notes supports all the functions of a scientific calculator, making it a versatile tool for students and professionals. Tapping the equals sign twice allows users to repeat the most recent mathematical function.

Other Notable Features in iOS 18 and iPadOS 18

Beyond the major highlights, iOS 18 and iPadOS 18 include numerous other improvements and new features across various apps and system functions:

• Activation Lock for iPhone Parts: Building on the existing Activation Lock that prevents unauthorized use of a stolen iPhone, this feature extends the lock to individual iPhone parts. The intention is to deter theft for parts, though concerns have been raised about potential complications for legitimate repairs, as highlighted in discussions about the iPhone 16 battery replacement process.
• Mail App Enhancements: The Mail app adopts a more organized layout reminiscent of Gmail, with new categories like Primary, Transactions, Updates, and Promotions. A "Digest" view groups all emails from a single sender, simplifying the process of catching up on communications from frequent contacts or companies.
• Safari Highlights: Safari gains a new Highlights feature that intelligently surfaces key information on a webpage, such as contact details, addresses, or article summaries, reducing the need to scroll extensively or click through multiple sections. Reader mode is also improved with the addition of a table of contents and summary view.
• Apple Maps Updates: Apple Maps now includes detailed topographic maps with trail networks and hiking information, including support for US National Parks. These maps can be saved for offline use, and users can create and save their own hiking routes.
• Apple Wallet Person-to-Person Payments: A convenient new feature in Apple Wallet allows users to send cash to another person by simply tapping two iPhones together or bringing them within close proximity. This eliminates the need to exchange phone numbers or email addresses for quick payments. A new "Preauthorized Payments" section provides a clear overview of upcoming payments for subscriptions and recurring bills.
• Game Mode: Borrowed from macOS, Game Mode arrives on iPhone and iPad. When activated during gaming, it minimizes background activity to prioritize performance and boost frame rates. It also reduces audio latency for AirPods and input lag for wireless controllers, providing a smoother gaming experience.
• AirPods Pro Features: The second-generation AirPods Pro and the new AirPods 4 gain Voice Isolation, which enhances voice quality during calls by reducing background noise and wind. Hands-free Siri interactions are also introduced, allowing users to respond to Siri prompts or accept/reject calls by nodding or shaking their head.
• AirPods Max Lossless Audio: Apple's over-ear headphones now support lossless audio up to 24-bit/48 kHz. This update also brings "ultra-low latency" for improved gaming performance and enhanced fidelity and response time for music production.
• Notes App Improvements: The Notes app adds live audio transcriptions that are searchable, making it easier to find specific information within recordings. Collapsible section headers help organize longer notes, and text can now be highlighted with various colors.
• Apple TV App Enhancements: A new InSight feature in the Apple TV app provides details about actors on screen and identifies music playing within a show or movie. Users can also now make purchases within the app on non-Apple devices using their Apple ID, with confirmation required via their iPhone.
• SharePlay Screen Control: SharePlay gains the ability to remotely control another user's screen, facilitating remote assistance or collaboration. Users can also draw on the shared screen to provide visual guidance.
• Accessibility Features: iOS 18 and iPadOS 18 introduce an eye-tracking mode, allowing users to control their iPhone or iPad using only their eyes, a significant accessibility advancement.
• Camera App Updates: iPhone 16 models gain the ability to switch to the front-facing camera using the Camera Control button. iPhone 15 Pro and 15 Pro Max users now have access to spatial photo capture (in addition to spatial video). Users can also lock auto exposure and autofocus settings. A "Require Screen On" feature ensures the Camera app only launches via the Camera Control button when the display is active.
• Find My Shared AirTags: The Find My app introduces a Lost AirTag section that allows users to share the location of an AirTag with up to five other people, aiding in tracking down lost items. A specific link can be shared via various apps, requiring authentication via an Apple account and expiring after a week. Support for sharing AirTag location with airlines is also planned.
• Vision Pro Companion App: A new stand-alone app for Vision Pro users is available on iPhone, allowing them to browse and download content, including apps and games, and manage their watch list when not wearing the headset.
• New Emoji: The emoji collection expands with eight new additions, including splatter, root vegetable, leafless tree, face with bags under eyes, fingerprint, harp, shovel, and a flag of Sark.
• Apple News+ Food Section: Subscribers to Apple News+ gain a new Food section featuring articles on restaurants, healthy eating, and a Recipe Catalog for browsing and saving recipes.
• Screen Time Notifications: Parents using Screen Time restrictions will now receive a notification when the passcode is used to override the set limits.
• CarPlay Ultra: Apple is rolling out the next generation of CarPlay, starting with Aston Martin vehicles. Known as CarPlay Ultra, this version extends the CarPlay experience beyond the main infotainment screen to integrate with the instrument cluster, providing a more cohesive and immersive in-car interface. More details are available here.

These updates collectively represent a significant evolution for Apple's mobile platforms. While Apple Intelligence takes center stage, promising a more intuitive and capable user experience powered by AI, the numerous other features, from enhanced customization and privacy controls to improved core apps and cross-device functionality, ensure that iOS 18 and iPadOS 18 offer substantial improvements for a wide range of users, even those without the latest AI-compatible hardware.