Baseline: tree as shared

This commit is contained in:
alexm committed 2026-10-05 03:03:38 +00:00
commit 31f2fb8a5a
1679 files changed
+1024562

No files matched your search

@@ -0,0 +1,7 @@
from std.python import Python
def main() raises:
# Create a mixed-type Python list
var py_list = Python.list(42, "cat", 3.14159)
for py_obj in py_list: # Each element is of type "PythonObject"
print(py_obj)
@@ -0,0 +1,7 @@
from std.python import Python
def main() raises:
# Create a mixed-type Python dictionary
var py_dict = Python.evaluate("{'a': 1, 'b': 2.71828, 'c': 'sushi'}")
for py_key in py_dict: # Each key is of type "PythonObject"
print(py_key, py_dict[py_key])
@@ -0,0 +1,7 @@
from std.python import Python
def main() raises:
# Create a mixed-type Python dictionary
var py_dict = Python.evaluate("{'a': 1, 'b': 2.71828, 'c': 'sushi'}")
for py_tuple in py_dict.items(): # Each 2-tuple is of type "PythonObject"
print(py_tuple[0], py_tuple[1])
@@ -0,0 +1,101 @@
# These are equivalent
raise Error("file not found")
raise "file not found"
```
The string literal form is a convenience—the compiler automatically wraps it in
an `Error`.
By declaring `raises`, you tell Mojo that a function may raise an error:
```mojo
def read_file_fn(path: String) raises -> String:
if not path:
raise "path cannot be empty"
return "contents of " + path
```
:::tip
If you need structured error data—like separate fields for an error code and
description—or allocation-free errors for GPU kernels, see
[Typed errors](#typed-errors) later on this page. For most application code,
`Error` with a descriptive string message is all you need.
:::
## Handle an error
Mojo uses `try`/`except` to detect and handle errors. The full syntax is:
```mojo
try:
# Code that might raise an error
except e:
# Runs if an error occurs
else:
# Runs if no error occurs
finally:
# Always runs, regardless of outcome
```
You must include one or both of `except` and `finally`. The `else` clause is
optional.
### How each clause works
- `try` — Contains code that might raise an error. If no error occurs, the
entire block executes. If an error occurs, execution stops at the `raise`
point and continues with the `except` clause (if present) or the `finally`
clause.
- `except` — Runs only when an error occurs in the `try` block. If you
provide a variable name (`except e:`), the error is bound to that variable.
A `try` block can have only one `except` clause.
- `else` — Runs only when no error occurs in the `try` block. The `else`
clause is *skipped* if the `try` clause exits via `continue`, `break`, or
`return`.
- `finally` — Runs after the `try` and any `except` or `else` clause,
regardless of outcome. It executes even if another clause exits via
`continue`, `break`, `return`, or by raising a new error. Use `finally` to
release resources (such as file handles) that must be cleaned up regardless
of whether an error occurred.
### Example
The following example demonstrates all four clauses. The `process_record()`
function raises `Error` for different conditions, and the caller loops over
a list of IDs to exercise each clause:
```mojo title="handle_error.mojo"
def process_record(id: Int) raises -> String:
if id < 0:
raise Error("invalid record ID: must be non-negative")
if id > 999:
raise Error("record not found")
return String("record_", id)
def main() raises:
try:
for id in [5, 0, 1001, -3, 42]:
var result: String
try:
print()
print("try => id:", id)
if id == 0:
continue
result = process_record(id)
except e:
if "invalid" in String(e):
print("except => fatal:", e)
raise e
print("except => handled:", e)
else:
print("else => success:", result)
finally:
print("finally => done with id:", id)
except e:
print("\nre-raised error:", e)
@@ -0,0 +1,91 @@
def error_func() raises -> Int:
raise "something went wrong"
def typed_func() raises ValidationError -> Int:
raise ValidationError(field="x", reason="invalid")
# This doesn't compile
def mixed() raises ValidationError:
try:
_ = error_func() # raises Error
_ = typed_func() # raises ValidationError
except e:
print(e)
```
The compiler reports:
```output
error: cannot call function that may raise 'Error' in a context that
supports an error type of 'ValidationError'
```
To call both functions, use separate `try` blocks or wrap the `Error`-raising
function as shown in
[Wrap `Error` at API boundaries](#wrap-error-at-api-boundaries).
### Recommendations for mixed codebases
When working with both `Error` and typed errors:
- **Use `raises YourErrorType`** — Always specify the error type in function
signatures. Bare `raises` discards type information.
- **Use separate `try` blocks** — When calling functions with different error
types, use nested or sequential `try` blocks to handle each type
independently.
:::note
For a complete working example of these interaction patterns, see
[`error_interaction.mojo`](https://github.com/modular/modular/blob/main/Mojo/docs/site/code/manual/errors/error_interaction.mojo).
:::
## Enable stack trace generation for errors
Because Mojo represents errors as alternate return values rather than
stack-unwinding exceptions, stack trace collection isn't automatic. Collecting
a stack trace requires heap allocation and adds runtime overhead, so it's
disabled by default to keep error handling lightweight.
:::important
Stack traces are a feature of the built-in `Error` type only. Typed errors
currently don't capture stack traces because the trace is collected inside
`Error.__init__()`, and custom error structs have no equivalent hook. The
examples in this section all use `Error` intentionally.
:::
Mojo generates a stack trace when your program hits a segmentation fault.
However, by default Mojo *doesn't* generate a stack trace when your program
raises an error—this avoids the additional runtime overhead. To enable stack
traces for raised errors, set the `MODULAR_DEBUG` environment variable to
`stack-trace-on-error`, as shown in the examples below.
Keep in mind that when you compile your program with
[`mojo build`](/docs/cli/build/), the compiler optimizes and strips symbols by
default, so often your stack trace won't be very useful.
Consider this program:
```mojo title="stacktrace_error.mojo"
def func2() raises -> None:
raise Error("Intentional error")
def func1() raises -> None:
func2()
def main() raises:
func1()
```
If you compile the program with default settings and run it with the
environment variable set, you'll see a stack trace without symbols:
```sh
mojo build stacktrace_error.mojo
```
```sh
MODULAR_DEBUG=stack-trace-on-error ./stacktrace_error
@@ -0,0 +1,29 @@
def func2() raises -> None:
raise Error("Intentional error")
def func1() raises -> None:
func2()
def main() raises:
try:
func1()
except e:
print(e)
print("-" * 20)
var stack_trace = e.get_stack_trace()
if stack_trace:
print(stack_trace.value())
else:
print("No stack trace available")
```
When you compile with debug symbols and run with stack trace generation enabled:
```sh
mojo build --debug-level full stacktrace_error_capture.mojo
```
```sh
MODULAR_DEBUG=stack-trace-on-error ./stacktrace_error_capture
@@ -0,0 +1,141 @@
# Obtain a file handle to read from storage
var f = open(input_file, "r")
var content = f.read()
# Process the content as needed
# Close the file handle
f.close()
```
Calling [`close()`](/docs/std/io/file/FileHandle/#close) releases the
memory and other operating system resources associated with the opened file. If
your program were to open many files without closing them, you could exhaust the
resources available to your program and cause errors. The problem is even worse
if you were writing to a file instead of reading from it, because the operating
system might buffer the output in memory until the file is closed. If your
program were to crash instead of exiting normally, that buffered data could be
lost instead of being written to storage.
The example above includes the call to `close()`, but it ignores the
possibility that [`read()`](/docs/std/io/file/FileHandle/#read) could
raise an error, which would prevent the `close()` from executing.
To handle this scenario, you could rewrite the code to use `try` like this:
```mojo
# Obtain a file handle to read from storage
var f = open(input_file, "r")
try:
var content = f.read()
# Process the content as needed
finally:
# Ensure that the file handle is closed even if read() raises an error
f.close()
```
However, the [`FileHandle`](/docs/std/io/file/FileHandle/) struct
returned by [`open()`](/docs/std/io/file/open/) is a context manager.
When used with Mojo's `with` statement, a context manager ensures that the
resources it manages are properly released at the end of the block, even if an
error occurs. In the case of a `FileHandle`, that means the call to `close()`
takes place automatically. So you could rewrite the example above to take
advantage of the context manager (and omit the explicit call to `close()`)
like this:
```mojo
with open(input_file, "r") as f:
var content = f.read()
# Process the content as needed
```
The `with` statement also allows you to use multiple context managers within the
same code block. As an example, the following code opens one text file, reads
its entire content, converts it to upper case, and then writes the result to a
different file:
```mojo
with open(input_file, "r") as f_in, open(output_file, "w") as f_out:
var input_text = f_in.read()
var output_text = input_text.upper()
f_out.write(output_text)
```
`FileHandle` is perhaps the most commonly used context manager. Other examples
of context managers in the Mojo standard library are
[`NamedTemporaryFile`](/docs/std/tempfile/tempfile/NamedTemporaryFile/),
[`TemporaryDirectory`](/docs/std/tempfile/tempfile/TemporaryDirectory/),
[`BlockingScopedLock`](/docs/std/utils/lock/BlockingScopedLock/), and
[`assert_raises`](/docs/std/testing/testing/assert_raises/). You can also
create your own custom context managers, as described in [Write a custom context
manager](#write-a-custom-context-manager) below.
## Write a custom context manager
Writing a custom context manager is a matter of defining a
[struct](/docs/manual/structs/) that implements two special *dunder* methods
("double underscore" methods): `__enter__()` and `__exit__()`:
- `__enter__()` is called by the `with` statement to enter the runtime context.
The `__enter__()` method should initialize any state necessary for the context
and return the context manager.
- `__exit__()` is called when the `with` code block completes execution, even if
the `with` code block terminates with a call to `continue`, `break`, or
`return`. The `__exit__()` method should release any resources associated with
the context. After the `__exit__()` method returns, the context manager is
destroyed.
If the `with` code block raises an error, then the `__exit__()` method runs
before any error processing occurs (that is, before it is caught by a
`try`/`except` structure or your program terminates). If you'd like to define
conditional processing for error conditions in a `with` code block, you can
implement an overloaded version of `__exit__()` that takes an error
argument. For more information, see
[Define a conditional `__exit__()` method](#define-a-conditional-__exit__-method)
and
[Handle typed errors in `__exit__()`](#handle-typed-errors-in-__exit__)
below.
For context managers that don't need to release resources or perform other
actions on termination, you are not required to implement an `__exit__()`
method. In that case the context manager is destroyed automatically after the
`with` code block completes execution.
Here is an example of implementing a `Timer` context manager, which prints the
amount of time spent executing the `with` code block:
```mojo title="context_mgr.mojo"
import std.sys
import std.time
@fieldwise_init
struct Timer(ImplicitlyCopyable):
var start_time: Int
def __init__(out self):
self.start_time = 0
def __enter__(mut self) -> Self:
self.start_time = Int(time.perf_counter_ns())
return self
def __exit__(mut self):
var end_time = time.perf_counter_ns()
var elapsed_time_ms = round(
Float64(end_time - self.start_time) / 1e6, 3
)
print("Elapsed time:", elapsed_time_ms, "milliseconds")
def main() raises:
with Timer():
print("Beginning execution")
time.sleep(1.0)
if len(sys.argv()) > 1:
raise "simulated error"
time.sleep(1.0)
print("Ending execution")
```
Running this example produces output like this:
```sh
mojo context_mgr.mojo
@@ -0,0 +1,169 @@
def __exit__(self, error: Error) raises -> Bool
```
Given the `Error` that occurred as an argument, the method can do any of the
following:
- Return `True` to suppress the error.
- Return `False` to re-raise the error.
- Raise a new error.
The following is an example of a context manager that suppresses only a certain
error condition and propagates all others:
```mojo title="conditional_context_mgr.mojo"
import std.time
@fieldwise_init
struct ConditionalTimer(ImplicitlyCopyable):
var start_time: Int
def __init__(out self):
self.start_time = 0
def __enter__(mut self) -> Self:
self.start_time = Int(time.perf_counter_ns())
return self
def __exit__(mut self):
var end_time = time.perf_counter_ns()
var elapsed_time_ms = round(
Float64(end_time - self.start_time) / 1e6, 3
)
print("Elapsed time:", elapsed_time_ms, "milliseconds")
def __exit__(mut self, e: Error) -> Bool:
if String(e) == "just a warning":
print("Suppressing error:", e)
self.__exit__()
return True
else:
print("Propagating error")
self.__exit__()
return False
def flaky_identity(n: Int) raises -> Int:
if (n % 4) == 0:
raise "really bad"
elif (n % 2) == 0:
raise "just a warning"
else:
return n
def main() raises:
for i in range(1, 9):
with ConditionalTimer():
print("\nBeginning execution")
print("i =", i)
time.sleep(0.1)
if i == 3:
print("continue executed")
continue
var j = flaky_identity(i)
print("j =", j)
print("Ending execution")
```
Running this example produces this output:
```output
Beginning execution
i = 1
j = 1
Ending execution
Elapsed time: 105.0 milliseconds
Beginning execution
i = 2
Suppressing error: just a warning
Elapsed time: 106.0 milliseconds
Beginning execution
i = 3
continue executed
Elapsed time: 106.0 milliseconds
Beginning execution
i = 4
Propagating error
Elapsed time: 106.0 milliseconds
Unhandled exception caught during execution: really bad
```
### Handle typed errors in `__exit__()`
The `__exit__(self, error: Error)` overload handles only `Error` values.
To handle typed errors, implement a parameterized `__exit__()` method with a
compile-time error type parameter:
```mojo
def __exit__[ErrType: AnyType](self, err: ErrType) -> Bool
```
This method receives the typed error directly, preserving its full type
information. You can use
[reflection](/docs/manual/metaprogramming/reflection/) to inspect the error
type at compile time. For example, you can:
- `reflect[ErrType].name()` — Get the error type's name as a string.
- `comptime if conforms_to(ErrType, Writable)` — Check if the error
implements `Writable`, and if so, access the error through its `Writable`
interface.
The following `ResourceGuard` example demonstrates this pattern:
```mojo
from std.reflection import *
@fieldwise_init
struct ConnectionError(Copyable, Writable):
var message: String
def write_to(self, mut writer: Some[Writer]):
writer.write("ConnectionError: ", self.message)
struct ResourceGuard(ImplicitlyCopyable):
var name: String
var suppress_errors: Bool
def __init__(out self, name: String, suppress_errors: Bool = False):
self.name = name
self.suppress_errors = suppress_errors
def __enter__(self) -> Self:
print("Acquiring:", self.name)
return self
def __exit__(self):
print("Releasing:", self.name, "(no error)")
def __exit__[ErrType: AnyType](self, err: ErrType) -> Bool:
comptime type_name = reflect[ErrType].name()
print("Releasing:", self.name)
print(" Error type:", type_name)
comptime if conforms_to(ErrType, Writable):
print(" Message:", err)
return self.suppress_errors
```
When no error occurs, `__exit__(self)` runs as usual. When a typed error occurs,
`__exit__[ErrType]()` runs instead, giving you access to the error type and
its data:
```mojo
# No error — calls __exit__(self)
with ResourceGuard("database"):
print("Working...")
# Typed error, suppressed — __exit__[ErrType] returns True
with ResourceGuard("cache", suppress_errors=True):
use_connection() # raises ConnectionError
print("Continued after suppressed error")
@@ -0,0 +1,61 @@
def multiplier[factor: Int](x: Int) -> Int:
return x * factor
def main():
comptime times_ten = multiplier[10]
var x10 = times_ten(3)
```
Parameters accept both types and values at compile time. When a
parameter accepts a type, the result is type-parameterized code. When it
accepts a value, the result is value-parameterized code.
```mojo no-test
struct MyList[T: AnyType]:
# ... type-parameterized
struct FixedBuffer[size: Int]:
# ... value-parameterized
```
Mojo's parameters are similar to C++ template parameters or Rust generic
parameters.
In Mojo, "parameter" always means a compile-time value, and
"argument" always means a run-time value.
In most other languages, a parameter is part of a declaration and an
argument is the value you pass at the call site. Mojo changes the
meaning of "parameter" to refer specifically to compile-time values.
Mojo makes this distinction visible in syntax: use `[]` for
parameters and `()` for arguments.
In addition to parameterizing structs and functions, you can also define
parameterized `comptime` values.
## Parameterized functions {#parameters-and-generics}
To define a *parameterized function*, add parameters in square brackets ahead
of the argument list. Each parameter is formatted just like an argument: a
parameter name, followed by a colon and a type. In the
following example, the function has a single parameter, `count` of type `Int`.
```mojo
def repeat[count: Int](msg: String):
comptime for i in range(count):
print(msg)
```
The [`comptime`](/docs/manual/metaprogramming/comptime-evaluation/#comptime-for)
keyword shown here
causes the `for` loop to be fully unrolled at compile time. The `comptime for`
requires the loop limits to be known at compile time. Since `count` is a
parameter, `range(count)` can be calculated at compile time.
Calling a parameterized function, you provide values for the parameters, just
like function arguments:
```mojo
repeat[3]("Hello")
@@ -0,0 +1,183 @@
def repeat_3(msg: String):
print(msg)
print(msg)
print(msg)
```
:::note
This doesn't represent actual code generated by the compiler. By the
time parameters are resolved, Mojo code has already been transformed to an
intermediate representation in [MLIR](https://mlir.llvm.org/).
:::
If the compiler can't resolve all parameter values to constant values,
compilation fails.
### Overloading on parameters
Functions and methods can be overloaded on their parameter signatures. For
information on overload resolution, see
[Overloaded functions](/docs/manual/functions/#overloaded-functions).
## Parameters at a glance
Parameters to a function or struct appear in square brackets after a function
or struct name. Parameters always require type annotations.
When you're looking at a function or struct signature, you may
see some special characters such as `/` and `*` in the parameter list.
Here's an example:
```mojo
def my_sort[
# infer-only parameters
dtype: DType,
width: SIMDLength,
//,
# positional-only parameter
values: SIMD[dtype, width],
/,
# positional-or-keyword parameter
compare: def(Scalar[dtype], Scalar[dtype]) thin -> Int,
*,
# keyword-only parameter
reverse: Bool = False,
]() -> SIMD[dtype, width]:
```
Here, `compare` is a function-typed parameter. Because the comparator is a
noncapturing function value, the function type explicitly uses `thin`.
Here's a quick overview of the special characters in the parameter list:
- Double slash (`//`): parameters declared before the double slash are
[infer-only parameters](#infer-only-parameters).
- Slash (`/`): parameters declared before a slash are positional-only
parameters. Positional-only and keyword-only parameters follow the same rules
as [positional-only and keyword-only
arguments](/docs/manual/functions#positional-only-and-keyword-only-arguments).
- A parameter name prefixed with a star, like `*Types` identifies a
[variadic parameter](#variadic-parameters) (not shown in the example above).
Any parameters following the variadic parameter are keyword-only.
- Star (`*`): in a parameter list with no variadic parameter, a star by itself
indicates that the following parameters are keyword-only parameters.
- An equals sign (`=`) introduces a default value for an
[optional parameter](#optional-parameters-and-keyword-parameters).
## Parameterized declarations
Parameterization let functions work across multiple types, and let containers
store values of many types. For example,
[`List`](/docs/std/collections/list/List/) takes a type parameter, so
`List[Int]` holds integers and `List[String]` holds strings.
In Mojo, parameterizations use compile-time elements. A function parameterized
on a type is type-parameterized. A function parameterized on a value is
value-parameterized. Both use `[]`.
This function uses both a type parameter and a value parameter:
```mojo
def repeat[
MsgType: Writable, // infer-only
count: Int
](msg: MsgType):
comptime for _ in range(count):
print(msg)
def main():
repeat[2](42) # prints 42 on two lines
```
`MsgType` is type-parameterized. It accepts any `Writable` type. `count` is
value-parameterized. It accepts a compile-time integer. Together, they let
you write one function that works across types and specializes for
different repeat counts.
`MsgType` uses `//` to mark it as an
[infer-only parameter](#infer-only-parameters). The compiler infers
the type from `msg`, so you only pass `count` explicitly.
For more on parameterized declarations, including trait conformance, conditional
conformance, and value parameterizations, see
[parameterized declarations](/docs/manual/generics/).
## Parameterized structs
You can also add parameters to structs. You can use parameterized structs to
build parameterized collections. For example, a parameterized array type might
include code like this:
```mojo
from std.memory.alloc import alloc, dealloc, ThinAllocation, Layout
struct ParameterizedArray[T: Copyable & Deinitable](
Writable where conforms_to(T, Writable)
):
var _data: ThinAllocation[Self.T]
var _size: Int
def __init__(out self, var *elements: Self.T):
self._size = len(elements)
self._data = alloc[Self.T]({count = self._size}).into_thin()
var ptr = self._data.unsafe_ptr()
for i in range(self._size):
ptr.unsafe_offset(i).unsafe_write(elements[i].copy())
def __init__(out self, *, count: Int, value: Self.T):
self._size = count
self._data = alloc[Self.T]({count = count}).into_thin()
var ptr = self._data.unsafe_ptr()
for i in range(self._size):
ptr.unsafe_offset(i).unsafe_write(copy=value)
def __deinit__(deinit self):
var ptr = self._data.unsafe_ptr()
for i in range(self._size):
ptr.unsafe_offset(i).unsafe_deinit_pointee()
dealloc(self._data^.unsafe_with_layout({count = self._size}))
def __getitem__(self, i: Int) raises -> ref[self] Self.T:
if i < self._size:
return self._data.unsafe_ptr().unsafe_origin_cast[
origin_of(self)
]()[unsafe_offset=i]
else:
raise Error("Out of bounds")
def write_to(
self, mut writer: Some[Writer]
) where conforms_to(Self.T, Writable):
writer.write("[")
var ptr = self._data.unsafe_ptr()
for i in range(self._size):
writer.write(ptr[unsafe_offset=i])
if i < self._size - 1:
writer.write(", ")
writer.write("]")
```
This struct has a single parameter, `T`, which is a placeholder for
the data type you want to store in the array, sometimes called a *type
parameter*. `T` conforms to the
[`Copyable`](/docs/std/traits/copyable/Copyable/) trait and therefore to the
[`Movable`](/docs/std/traits/movable/Movable/) trait.
As with parameterized functions, you need to pass in parameter values when you
use a parameterized struct. In this case, when you create an instance of
`ParameterizedArray`, you need to specify the type you want to store, like
`Int`, or `Float64`. (This is a little confusing, because the *parameter value*
you're passing in this case is a *type*. That's OK: a Mojo type is a valid
compile-time value.)
You'll see that `Self.T` is used throughout the struct where you'd
usually see a type name. For example, as the formal type for the `elements` in
the initializer, and the return type of the `__getitem__()` method.
Here's an example of using `ParameterizedArray`:
```mojo
var array = ParameterizedArray(1, 2, 3)
print(array)
@@ -0,0 +1,33 @@
# Make a vector of 4 floats.
var small_vec = SIMD[DType.float32, 4](1.0, 2.0, 3.0, 4.0)
# Make a big vector containing 1.0 in float16 format.
var big_vec = SIMD[DType.float16, 32](1.0)
# Do some math and convert the elements to float32.
var bigger_vec = (big_vec + big_vec).cast[DType.float32]()
```
Note that the `cast()` method also needs a parameter to specify the type you
want from the cast (the method definition above expects a `target` parameter
value). Thus, just as the `SIMD` struct is a parameterized type definition, the
`cast()` method is a parameterized method definition. At compile time, the
compiler creates a concrete version of the `cast()` method with the target
parameter bound to `DType.float32`.
The code above shows the use of concrete types (that is, the parameters are all
bound to known values). But the major power of parameters comes from the ability
to define parameterized algorithms and types (code that uses the parameter
values). For example, here's how to define a parameterized algorithm with
`Scalar` that is datatype agnostic:
```mojo
from std.math import sqrt
def rsqrt[dt: DType](x: Scalar[dt]) -> Scalar[dt]:
return 1 / sqrt(x)
def main():
var v = Scalar[DType.float16](42)
print(rsqrt(v))
@@ -0,0 +1,85 @@
struct Span[mut: Bool, //, T: Copyable, origin: Origin[mut]]:
# ... implementation omitted
```
Here, the `mut` parameter is infer-only. The value is usually inferred when you
create an instance of `Span`. Binding the `mut` parameter by keyword lets you
define a `Span` that requires a mutable origin.
```mojo
def mutate_span(span: Span[mut=True, Byte, _]):
for i in range(0, len(span), 2):
if i + 1 < len(span):
span.swap_elements(i, i + 1)
```
If the compiler can't infer the value of an infer-only parameter, and it's not
specified by keyword, compilation fails.
## Parameter expressions are just Mojo code
A parameter expression is any code expression (such as `a+b`) that occurs where
a parameter is expected. Parameter expressions support operators and function
calls, just like run-time code, and all parameter types use the same type
system as the run-time program (such as `Int` and `DType`).
Because parameter expressions use the same grammar and types as run-time
Mojo code, you can use many
["dependent type"](https://en.wikipedia.org/wiki/Dependent_type) features. For
example, you might want to define a helper function to concatenate two SIMD
vectors:
```mojo
def concat[
dtype: DType, ls_size: Int, rh_size: Int, //
](lhs: SIMD[dtype, ls_size], rhs: SIMD[dtype, rh_size]) -> SIMD[
dtype, ls_size + rh_size
]:
var result = SIMD[dtype, ls_size + rh_size]()
comptime for i in range(ls_size):
result[i] = lhs[i]
comptime for j in range(rh_size):
result[ls_size + j] = rhs[j]
return result
```
Note that the resulting length is the sum of the input vector lengths, and a
simple `+` operation expresses this.
### Powerful compile-time programming
While simple expressions are useful, sometimes you want to write imperative
compile-time logic with control flow. You can even do compile-time recursion.
For instance, here is an example "tree reduction" algorithm that sums all
elements of a vector recursively into a scalar:
```mojo
def slice[
dtype: DType, size: Int, //
](x: SIMD[dtype, size], offset: Int) -> SIMD[dtype, size // 2]:
comptime new_size = size // 2
var result = SIMD[dtype, new_size]()
for i in range(new_size):
result[i] = Scalar[dtype](x[i + offset])
return result
def reduce_add(x: SIMD) -> Int:
comptime if x.length == 1:
return Int(x[0])
elif x.length == 2:
return Int(x[0]) + Int(x[1])
# Extract the top/bottom halves, add them, sum the elements.
comptime half_size = x.length // 2
var lhs = slice(x, 0)
var rhs = slice(x, half_size)
return reduce_add(lhs + rhs)
def main():
var x = SIMD[DType.int, 4](1, 2, 3, 4)
print(x)
print("Elements sum:", reduce_add(x))
@@ -0,0 +1,294 @@
struct Pointer[
mut: Bool,
//,
T: AnyType,
origin: Origin[mut=mut],
*,
address_space: AddressSpace = .GENERIC,
]
```
For pointers initialized with the `to` keyword argument, the origin is inferred
from the origin of the pointee. For example, in the following code,
`s_ptr.origin` is the same as the origin of `s`:
```mojo
var s = "Testing"
var s_ptr = Pointer(to=s)
```
When allocating memory with the `alloc()` function, the returned pointer has an
`origin` value of `MutUntrackedOrigin`. This value represents an origin that is
mutable and doesn't *alias* existing values. For example, it doesn't point to
the memory allocated for any other variable. This memory isn't
tracked by Mojo's lifetime checker and you're responsible for freeing it.
If you're using a pointer in the implementation of a struct, you usually
don't have to worry about the origin, as long as the pointer isn't exposed
outside of the struct. For example, if you implement a static array type
that allocates memory in its initializer, deallocates in its deinitializer,
and doesn't expose the pointer outside of the struct, the default origin is
fine.
But if the struct exposes a pointer or reference to that memory, you need
to set the origin appropriately. For example, the
[`List`](/docs/std/collections/list/List/) type has an `unsafe_ptr()`
method that returns a `Pointer` to the underlying storage. In this case,
the returned pointer should share the origin of the list, since the list is
the logical owner of the storage.
That method looks something like this:
```mojo no-test
def unsafe_ptr[
origin: Origin, address_space: AddressSpace, //
](ref[origin, address_space] self) -> Pointer[
Self.T, origin, address_space=address_space
]:
return (
self._data.unsafe_mut_cast[origin.mut]()
.unsafe_origin_cast[origin]()
.unsafe_address_space_cast[address_space]()
)
```
This returns a copy of the original pointer, with the origin set to match the
origin and mutability of the `self` value.
A method like this is unsafe, but setting the correct origin makes it safer,
since the compiler knows that the pointer is referring to data owned by the
list.
When taking a pointer as a function argument, you often want to require either
a mutable or immutable origin, but otherwise allow the compiler to infer the
origin. Here's an example:
```mojo
def print_bytes(bytes: Pointer[mut=False, Byte, _], count: Int):
for i in range(count):
print(hex(bytes[unsafe_offset=i]), end=" ")
print()
```
By binding the infer-only `mut` parameter to `False`, and leaving the origin
unbound (using `_`), this signature lets the compiler infer the origin, but
forces the origin to be immutable. Mojo can implicitly cast a mutable pointer to
an immutable pointer, so you can pass a mutable pointer into `print_bytes()`,
but the function can't mutate the data.
## Working with nullability
`Pointer` is a non-nullable type. To model a null pointer, wrap it
in [`Optional`](/docs/std/collections/optional/Optional/):
```mojo
var ptr = Optional[Pointer[Int, MutUntrackedOrigin]]()
```
This creates an `Optional` with a value of `None`, which is equivalent
to a null pointer. `Optional[Pointer]` has the same memory layout
as a raw pointer, so you can pass it across FFI boundaries as `NULL`.
To check whether an optional pointer is null, use `Optional` methods:
```mojo
if ptr:
# ptr is not None — safe to unwrap
var p = ptr.value()
```
When you need a non-null value for deferred initialization, use
`unsafe_dangling()` instead of an `Optional`:
```mojo
var ptr = Pointer[Int, MutUntrackedOrigin].unsafe_dangling()
```
For a practical example of optional pointers in a data structure, see
[Self-referential structs](/docs/manual/structs/reference/).
## More memory allocation patterns
In some cases, you may not want to hold on to an `Allocation`:
- When allocating data for a struct, you may want to use a
[`ThinAllocation`](/docs/std/memory/alloc/ThinAllocation/) instead, to avoid
using extra memory.
- When working with raising functions, you sometimes need to avoid
an explicitly-destroyed type like `Allocation` or `ThinAllocation`.
The following sections describe these special cases.
### Holding an allocation in a struct field
When storing an allocation as a struct field, you may not want to store the
extra layout data included in the `Allocation` struct. The layout data is two
`Int` values (alignment and element count), typically an extra 16 bytes per
allocation. If your struct already tracks the amount of space it's allocated,
you can eliminate this extra space by storing a `ThinAllocation`, which is an
explicitly-destroyed wrapper around a pointer.
The `Allocation.into_thin()` method consumes the original allocation and
returns a `ThinAllocation`:
```mojo
struct Counter:
comptime _layout = Layout[Int].single()
var _alloc: ThinAllocation[Int]
def __init__(out self, value: Int):
self._alloc = alloc(Self._layout).into_thin()
self._alloc.unsafe_ptr().unsafe_write(value)
def increment(mut self):
self._alloc.unsafe_ptr()[] += 1
def get(self) -> Int:
return self._alloc.unsafe_ptr()[]
def __deinit__(deinit self):
# Convert ThinAllocation back into Allocation
dealloc(self._alloc^.unsafe_with_layout(Self._layout))
```
To deallocate a `ThinAllocation`, you need to supply the original layout
to reconstruct an `Allocation` using the `unsafe_with_layout()` method.
This example shows storing the layout as a comptime member; for a struct
with a dynamic size, you can reconstruct the original layout:
```mojo no-test
self._alloc^.unsafe_with_layout({count = size})
```
### Allocations and raising functions
Because `Allocation` and `ThinAllocation` need to be explicitly deallocated
before they go out of scope, they can conflict with raising functions.
Consider the following code:
```mojo no-test
def allocating_function() raises:
var data = alloc[Float64]({count = 64})
# ...
raising_function(data.unsafe_ptr())
dealloc(data^)
```
Because an error can cause `allocating_function()` to exit without executing the
`dealloc()` call, the compiler identifies this as a potential memory leak. There
are a couple of approaches to this problem. The function can use a
`try`/`except` statement to ensure that the memory is deallocated in the event
of an error:
```mojo
def allocating_function() raises:
var data = alloc[Float64]({count = 64})
# ...
try:
raising_function(data.unsafe_ptr())
except e:
dealloc(data^)
raise e^ # propagate the error
dealloc(data^)
```
Where this isn't viable, the alternative is to use the
`unsafe_leak()` method to take ownership of the allocation's
pointer. This consumes the allocation, but requires you to
ensure the memory is deallocated. You should consider this
pattern a last resort if other patterns don't work:
```mojo
def leaky_function() raises:
var data_ptr = alloc[Float64]({count = 64}).unsafe_leak()
# ...
raising_function(data_ptr)
dealloc(
ThinAllocation(unsafe_owned_ptr=data_ptr).unsafe_with_layout(
{count = 64}
)
)
```
Downsides of this approach include:
- If `raising_function()` raises an error in this example, `dealloc()` never
gets called, leaking the memory.
- When you reconstruct an `Allocation` from a `Pointer` like this, you run the
risk of freeing the same memory twice.
## Working with foreign pointers
When exchanging data with other programming languages, you may need to construct
a `Pointer` from a foreign pointer. Mojo restricts creating
`Pointer` instances from arbitrary addresses, to avoid users accidentally
creating pointers that *alias* each other (that is, two pointers that refer to
the same location). However, there are specific methods you can use to get a
`Pointer` from a Python or C/C++ pointer.
When dealing with memory allocated elsewhere, you need to be aware of who's
responsible for freeing the memory. Freeing memory allocated elsewhere
can result in undefined behavior.
When working with some foreign functions, you may need to supply a pointer with
no specific type (a type-erased pointer, or "void pointer" in C/C++). This is
equivalent to a Mojo `OpaquePointer`.
You also need to be aware of the format of the data stored in memory, including
data types and byte order. For more information, see
[Converting data: bitcasting and byte order](#converting-data-bitcasting-and-byte-order).
### Creating a Mojo pointer from a raw memory address
You can create a `Pointer` from a raw memory address using the
`unsafe_from_address` initializer.
```mojo
def write_to_address(mmio_address: Int, value: Int32):
var ptr = Pointer[Int32, MutUntrackedOrigin](
unsafe_from_address=mmio_address
)
# Writing to a raw memory address may require a volatile load/store as the
# operation may have side effects not visible to the compiler.
# You can specify this using the `volatile` parameter.
ptr.unsafe_store[volatile=True](value)
```
This is unsafe, as the caller must ensure the address is valid before writing to
it, and that the memory is initialized before reading from it. The caller must
also ensure the pointer's origin and mutability are valid for the address;
failure to do so may result in undefined behavior.
### Creating a Mojo pointer from a Python pointer
The `PythonObject` type defines an
[`unsafe_get_as_pointer()`](/docs/std/python/python_object/PythonObject/#unsafe_get_as_pointer)
method to construct a `Pointer` from a Python address.
:::note
Where possible, use safer methods to exchange data with Python, such as the
[`python.numpy`](/docs/std/python/numpy/) module, which provides convenience
functions for transferring 1D NumPy arrays between Mojo and Python.
:::
The following code creates a NumPy array and then accesses the data
using a Mojo pointer:
```mojo
from std.python import Python
def share_array() raises:
var np = Python.import_module("numpy")
var arr = np.array(Python.list(1, 2, 3, 4, 5, 6, 7, 8, 9))
var ptr = arr.ctypes.data.unsafe_get_as_pointer[.int64]()
for i in range(9):
print(ptr[unsafe_offset=i], end=", ")
print()
def main() raises:
share_array()
@@ -0,0 +1,13 @@
from std.python import Python
def main() raises:
var py_module = """
def type_printer(value):
print(type(value))
"""
var py_utils = Python.evaluate(py_module, file=True, name="py_utils")
py_utils.type_printer(4)
py_utils.type_printer(3.14)
py_utils.type_printer(True)
py_utils.type_printer("Mojo")
@@ -0,0 +1,8 @@
from std.python import Python
def main() raises:
var py_dict = Python.dict()
py_dict["item_name"] = "whizbang"
py_dict["price"] = 11.75
py_dict["inventory"] = 100
print(py_dict)
@@ -0,0 +1,9 @@
from std.python import Python
def main() raises:
var py_list = Python.list("cat", 2, 3.14159, 4)
var n = py_list[2]
print("n =", n)
py_list.append(5)
py_list[0] = "aardvark"
print(py_list)
@@ -0,0 +1,7 @@
from std.python import Python
def main() raises:
var py_tuple = Python.tuple("cat", 2, 3.1415, "cat")
var n = py_tuple[2]
print("n =", n)
print("Number of cats:", py_tuple.count("cat"))
@@ -0,0 +1,8 @@
from std.python import Python
def main() raises:
var py_set = Python.evaluate('{2, 3, 2, 7, 11, 3}')
var num_items = len(py_set)
print(num_items, "items in the set.")
var contained = 7 in py_set
print("Is 7 in the set:", contained)
@@ -0,0 +1,49 @@
from std.python import Python
from std.python import PythonObject
def main() raises:
var py_string = PythonObject("Hello, Mojo!")
var py_bool = PythonObject(True)
var py_int = PythonObject(123)
var py_float = PythonObject(3.14)
var mojo_string = String(py=py_string)
var mojo_bool = Bool(py=py_bool)
var mojo_int = Int(py=py_int)
var mojo_float = Float64(py=py_float)
```
### Comparing Python types in Mojo
You can use Python objects in Mojo comparison expressions, and the Mojo `is`
operator also works to compare the identity of two Python objects. Python values
like `False` and `None` evaluate as false in Mojo boolean expressions as well.
If you need to know the type of the underlying Python object, you can use the
[`Python.type()`](/docs/std/python/python/Python/#type) method, which is
equivalent to the Python `type()` builtin. You can test if a Python
object is of a particular type by performing an identity comparison against the
type as shown below:
```mojo
from std.python import Python
def main() raises:
var value1 = PythonObject(3.7)
var value2 = Python.evaluate("10/3")
# Compare values
print("Is value1 greater than 3:", value1 > 3)
print("Is value1 greater than value2:", value1 > value2)
# Compare identities
var value3 = value2
print("value1 is value2:", value1 is value2)
print("value2 is value3:", value2 is value3)
# Compare types
var py_float_type = Python.evaluate("float")
print("Python float type:", py_float_type)
print("value1 type:", Python.type(value1))
print("Is value1 a Python float:", Python.type(value1) is py_float_type)
@@ -0,0 +1,226 @@
var s: String = "Testing"
s += " Mojo strings"
print(s) # Testing Mojo strings
```
### Construction
Many standard library types conform to the
[`Writable`](/docs/std/format/Writable/) trait, which indicates
that a value can be converted into a `String` using the `String(...)`
initializer.
The built-in [`print()`](/docs/std/io/io/print/) function accepts values
that conform to the `Writable` trait.
Use `String(value)` to explicitly convert a value to a `String`:
```mojo
var s = "Items in list: " + String(5)
print(s) # Items in list: 5
```
Or, use the string initializer with variadic `Writable` types,
so you don't have to call `String()` on each value:
```mojo
var s = String("Items in list: ", 5)
print(s) # Items in list: 5
```
### Emoji and grapheme clusters
Mojo source files are UTF-8, letting you write emoji and other non-ASCII
characters directly inside string literals.
```mojo
var wave = "👋"
```
The standard library counts emoji three different ways, and the answers
usually disagree:
- `byte_length()` returns the number of UTF-8 bytes.
- `count_codepoints()` counts the Unicode code points.
- `count_graphemes()` returns the number of user-perceived characters
(grapheme clusters), following
[UAX #29](https://www.unicode.org/reports/tr29/).
The grapheme count is what matches what a human would tell you if you
asked them to "count characters." A family emoji, a flag, and a waving
hand with a skin tone are each one grapheme, even though each is built
from several joined code points:
```mojo
def show(label: StaticString, s: StringSlice):
print(
label,
"bytes=", s.byte_length(),
"codepoints=", s.count_codepoints(),
"graphemes=", s.count_graphemes(),
)
def main():
show("family ", "👨‍👩‍👧‍👦")
show("flag ", "🇺🇸")
show("wave ", "👋🏽")
show("namaste ", "नमस्ते")
```
Output:
```text
family bytes=25 codepoints=7 graphemes=1
flag bytes=8 codepoints=2 graphemes=1
wave bytes=8 codepoints=2 graphemes=1
namaste bytes=18 codepoints=6 graphemes=3
```
If you'd rather not embed non-ASCII bytes in your source, for example, in
ASCII-only codebases, you can spell the code point with a hex escape or
`chr()`:
```mojo
var wave = "\U0001F44B" # 8-digit hex escape
var wave2 = chr(0x1F44B) # chr() function
var copy = "\u00A9" # 4-digit hex escape, ©
var euro = "\u20AC" # 4-digit hex escape, €
```
Mojo's 4-digit `\uHHHH` escape spells code points up to U+FFFF. The
8-digit `\U` form extends to U+10FFFF, Unicode's upper limit, covering
emoji like 👋 (U+1F44B) and other characters above the Basic Multilingual
Plane. The `chr()` function works for the full range too.
Both forms reject surrogate code points (U+D800 to U+DFFF). Surrogates
are reserved for UTF-16 encoding and aren't valid on their own. To
escape a character above U+FFFF, write its full code point with `\U`.
Don't use a UTF-16 surrogate pair.
### String formatting
The `format()` method inserts values into a string using manual or
automatic positional indexing. Replacement fields use braces:
```mojo
print("{0} {1} {0}".format("Mojo", 1.125)) # Mojo 1.125 Mojo
print("{} {}".format(True, "hello world")) # True hello world
```
Mojo's `TString` (template string) replaces the functionality of
the `format()` method with direct expressions and better
performance characteristics.
`TString`s work like `format()`, but they insert
[`Writable`](/docs/std/format/Writable/) representations of expressions
into replacement fields. This provides safe and flexible string
processing:
```mojo
var count = 3
var items = "apples"
var template = t"Give me {count} {items}." # Template string
print(template) # Output: Give me 3 apples.
```
`TString` values are lazy. They don't allocate until you explicitly
construct a `String`:
```mojo
var x = 41
print(t"The answer is {x + 1}") # The answer is 42 (no allocation)
var name = "Nate"
var template = t"Hello, {name}!" # template creation
print(template) # Hello, Nate! (no allocation)
var s = String(template) # explicitly construct a string
```
`TString`s can add arbitrary expressions within the replacement fields:
```mojo
var list: List[Int] = [1, 2, 3]
print(t"{list[0] + list[1]}") # 3
```
### Raw strings
In Mojo, **raw strings** are string literals prefixed with `r`.
If you ran the following command it would print on one line, not
two, because raw strings prevent backslash escape sequences from
being interpreted:
```mojo
print(r"Hello\nWorld") # Hello\nWorld, with the backslash and n
```
Raw strings help with regular expressions, code generation,
serialization code, and other applications where you want to use
escape sequences as literal entries in your string. Unlike
normal strings, escapes aren't processed.
Rawness applies to all forms of strings: single line, multi-line,
docstrings, and `TString`s. Raw `TString`s support interpolation
but won't expand escape sequences:
```mojo
var name = "Nate"
print(rt"Hello,\t{name}.") # Hello,\tNate., with the backslash and t
```
Raw strings *still* need a way to terminate, so if you have to use
`"` within your content, use an alternate form of quote, such as
single quote (`'`) or triple quotes, (`"""`, `'''`) to enclose
it:
```mojo
r'She said, "Hello, World!"'
```
### String literals
As with numeric types, the standard library includes a string literal type used
to represent literal strings in the program source. String literals are
enclosed in either single or double quotes.
Adjacent literals are concatenated together, so you can define a long string
using a series of literals broken up over several lines:
```mojo
comptime s = "A very long string which is "
"broken into two literals for legibility."
```
To define a multi-line string, enclose the literal in three single or double
quotes:
```mojo
comptime s = """
Multi-line string literals let you
enter long blocks of text, including
newlines."""
```
Note that the triple double quote form is also used for API documentation
strings.
A `StringLiteral` will materialize to a `String` when used at run-time:
```mojo
comptime param = "foo" # type = StringLiteral
var runtime_value = "bar" # type = String
var runtime_value2 = param # type = String
```
## Booleans
Mojo's `Bool` type represents a boolean value. It can take one of two values,
`True` or `False`. You can negate a boolean value using the `not` operator.
```mojo
var conditionA = False
var conditionB: Bool
conditionB = not conditionA
print(conditionA, conditionB)
@@ -0,0 +1,6 @@
def print_str(s: String):
print(s)
def main():
var name: String = "Joan"
print_str(name)
@@ -0,0 +1,329 @@
struct ImmutRef[origin: ImmOrigin]:
pass
```
Or you can use the [`Origin`](/docs/std/origin/Origin/)
struct to specify an origin with parametric mutability:
```mojo
struct ParametricRef[
is_mutable: Bool,
//,
origin: Origin[mut=is_mutable]
]:
pass
```
Origin types carry the mutability of a reference as a boolean parameter value,
indicating whether the origin is mutable, immutable, or even with mutability
depending on a parameter specified by the enclosing API.
The `is_mutable` parameter here is an [infer-only
parameter](/docs/manual/parameters/#infer-only-parameters). The `origin` value
is often inferred, as well. For example, the following code creates a
[`Pointer`](/docs/std/memory/pointer/Pointer/) to an existing value, but
doesn't need to specify an origin—the `origin` is inferred from the existing
value.
```mojo
from std.memory import Pointer
def use_pointer():
var a = 10
var ptr = Pointer(to=a)
```
### Origin sets
An `OriginSet` is not a type of origin, it represents a group of origins. Origin
sets are used for tracking the lifetimes of values captured in parametric
closures.
An `OriginSet` **isn't** a general-purpose mechanism for expressing a
combination of multiple origins. Instead, you can use `origin_of()` to express
an [origin union](#origin-unions).
### Origin values
Most origin values are created by the compiler. As a developer, there are a
few ways to specify origin values:
- Static origin. The `ImmStaticOrigin` `comptime` value
represents immutable values that last for the duration of the program.
String literal values have a `ImmStaticOrigin`.
- Derived origin. The `origin_of()` magic function returns the origin
associated with the value (or values) passed in.
- Inferred origin. You can use inferred parameters to capture the origin of a
value passed in to a function.
- Untracked origins. The untracked origins, `MutUntrackedOrigin` and
`ImmUntrackedOrigin` represent values that are not tracked by the lifetime
checker, such as dynamically-allocated memory.
- Wildcard origins. The `ImmUnsafeAnyOrigin` and `MutUnsafeAnyOrigin`
`comptime` values are special cases indicating a reference that might access
any live value.
#### Static origins
You can use the static origin `ImmStaticOrigin` when you have a
value that exists for the entire duration of the program.
For example, the `StringLiteral` method
[`as_string_slice()`](/docs/std/builtin/string_literal/StringLiteral/#as_string_slice)
returns a [`StringSpan`](/docs/std/collections/string/string_span/StringSpan/)
pointing to the original string literal. String literals are static—they're
allocated at compile time and never destroyed—so the slice is created with an
immutable, static origin.
#### Derived origins
Use the `origin_of(value)` operator to obtain a value's origin. An argument
to `origin_of()` can take an arbitrary expression that yields one of the
following:
- An origin value.
- A value with a memory location.
For example:
```mojo
origin_of(self)
origin_of(x.y)
origin_of(foo())
```
The `origin_of()` operator is analyzed statically at compile time;
The expressions passed to `origin_of()` are never evaluated. (For example,
when the compiler analyzes `origin_of(foo())`, it doesn't run the `foo()`
function.)
The following struct stores a string value using a
[`OwnedPointer`](/docs/std/memory/owned_pointer/OwnedPointer/): a smart
pointer that holds an owned value. The `as_ptr()` method returns a `Pointer` to
the stored string, using the same origin as the original `OwnedPointer`.
```mojo
from std.memory import OwnedPointer, Pointer
struct BoxedString:
var o_ptr: OwnedPointer[String]
def __init__(out self, value: String):
self.o_ptr = OwnedPointer(value)
def as_ptr(mut self) -> Pointer[String, origin_of(self.o_ptr)]:
return Pointer(to=self.o_ptr[])
```
Note that the `as_ptr()` method takes its `self` argument as `mut self`. If it
used the default argument convention, it would be immutable, and the
derived origin (`origin_of(self.o_ptr)`) would also be immutable.
You can also pass multiple expressions to `origin_of()` to express the union
of two or more origins:
`origin_of(a, b)`
#### Origin unions
When a function returns a reference or pointer that can have one of several
different origins, you can express the referenced origin as a union of all
of the possible origin values.
The union of two or more origins creates a new origin that references all of the
original origins for the purposes of lifetime extension (so a union of the
origins of `a` and `b` extends both lifetimes). An origin union is mutable if
and only if all of its constituent origins are mutable. Use an origin union
For an example, see
[Return values with union origins](#return-values-with-union-origins).
#### Inferred origins
Since origins are parameters, the compiler can *infer* an origin value from
the argument passed to a function or method, as described in
[Parameter inference](/docs/manual/parameters/#parameter-inference). This allows
a function to return a value that has the same origin as the argument passed to
it.
See the section on [`ref` arguments](#ref-arguments) for an example using an
inferred origin.
#### Untracked origins
The untracked origins, `MutUntrackedOrigin` and `ImmUntrackedOrigin` represent
values that do not alias any existing value. That is, they point to memory that
is not owned by any other variable, and are therefore not tracked by the
lifetime checker. For example, the
[`alloc()`](/docs/std/memory/alloc/alloc/) function returns an
`Allocation` for a new dynamically-allocated block of memory, with the origin
`MutUntrackedOrigin`. The origin indicates that the memory is not managed by the
Mojo ownership system. When you use an unsafe API like this, you're responsible
for managing the lifetime yourself: for example, a struct that allocates memory
should generally free that memory in its deinitializer.
#### Wildcard origins
The wildcard origins, `ImmUnsafeAnyOrigin` and `MutUnsafeAnyOrigin`, are
special cases indicating a reference that might access any live value. These
were previously widely used for unsafe pointers. Using a pointer with a wildcard
origin into a scope effectively disables Mojo's ASAP destruction for any values
in that scope, as long as the pointer is live. It also prevents Mojo from
enforcing
[argument exclusivity](/docs/manual/values/ownership/#argument-exclusivity) and
hides unused variable warnings.
Accordingly, the use of wildcard origins is discouraged, and should be used as a
last resort.
## Working with references
You can use the `ref` keyword with arguments and return values to specify a
reference with parametric mutability. That is, they can be either mutable or
immutable.
A `ref` return value looks like any other return value to the calling function,
but it's a *reference* to an existing value, not a copy.
### `ref` arguments
The `ref` argument convention lets you specify an argument of parametric
mutability: that is, you don't need to know in advance whether the passed
argument will be mutable or immutable. There are several reasons you might want
to use a `ref` argument:
- You want to accept an argument with parametric mutability.
- You want to tie the lifetime of one argument to the lifetime of another
argument.
- When you want an argument that is guaranteed to be passed in memory: this
can be useful for parameterized arguments that need an identity, whether
or not the concrete type is register passable.
The syntax for a `ref` argument is:
<code><strong>ref</strong> <var>arg_name</var>: <var>arg_type</var></code>
Or:
<code><strong>ref[</strong><var>origin_specifier(s)</var><strong>] </strong>
<var>arg_name</var>: <var>arg_type</var></code>
In the first form, the origin and mutability of the `ref` argument is inferred
from the value passed in. The second form includes an origin clause, consisting
of one or more origin specifiers inside square brackets. An origin
specifier can be either:
- An origin value.
- An arbitrary expression, which is treated as shorthand for
`origin_of(expression)`. In other words, the following declarations are
equivalent:
```mojo
ref[origin_of(self)]
ref[self]
```
- An [`AddressSpace`](/docs/std/memory/address_space/AddressSpace/) value.
- An underscore character (`_`) to indicate that the origin is *unbound*. This
is equivalent to omitting the origin specifier.
```mojo
def add_ref(ref a: Int, b: Int) -> Int:
return a+b
```
You can also name the origin explicitly. This is useful if you want to
restrict the argument to either a `ImmOrigin` or `MutOrigin`, or if you
want to bind a function's return value to the origin of an argument.
For example, the `Span` type is a non-owning view of contiguous data (like
a substring of a string, or a subset of a list). Because it points to data
that it doesn't own, it is parameterized on an origin value that represents
the lifetime and ownership of the data it points to.
In the following example, the `to_byte_span()` function takes a
`List[Byte]` and returns a `Span[Byte]` with the same origin as the list:
```mojo
from std.collections import List, Span
def to_byte_span[
is_mutable: Bool,
//,
origin: Origin[mut=is_mutable],
](ref[origin] list: List[Byte]) -> Span[Byte, origin]:
return Span(list)
def main():
var list: List[Byte] = [77, 111, 106, 111]
_ = to_byte_span(list)
```
In this example, the `origin` parameter is inferred from the `list` argument,
and then used as the origin for the returned `Span`.
Since the `Span` takes on the origin of the `list` argument, the Mojo compiler
can identify the span's data as owned by the list. The span will have the same
lifetime as the list, and the span will be mutable if the list is mutable.
### `ref` return values
Like `ref` arguments, `ref` return values allow a function to return a mutable
or immutable reference to a value. The syntax for a `ref` return value is:
<code><strong>-> ref[</strong><var>origin_specifier(s)</var><strong>] </strong>
<var>arg_type</var></code>
Note that you **must** provide an origin specifier for a `ref` return value. The
values allowed for origin specifiers are the same as the ones listed for
[`ref` arguments](#ref-arguments).
`ref` return values can be an efficient way to handle updating items in a
collection. The standard way to do this is by implementing the `__getitem__()`
and `__setitem__()` dunder methods. These are invoked to read from and write to
a subscripted item in a collection:
```mojo
var value = list[a]
list[b] += 10
```
With a `ref` argument, `__getitem__()` can return a mutable reference that can
be modified directly. This has pros and cons compared to using a `__setitem__()`
method:
- The mutable reference is more efficient—a single update isn't broken up across
two methods. However, the referenced value must be in memory.
- A `__getitem__()`/`__setitem__()` pair allows for arbitrary code to be run
when values are retrieved and set. For example, `__setitem__()` can validate
or constrain input values.
For example, in the following example, `NameList` has a `__getitem__()` method
that returns a reference:
```mojo
struct NameList:
var names: List[String]
def __init__(out self, *names: String):
self.names = []
for name in names:
self.names.append(name)
def __getitem__(ref self, index: Int) raises -> ref[self.names[0]] String:
if (index >=0 and index < len(self.names)):
return self.names[index]
else:
raise Error("index out of bounds")
def main() raises:
var list = NameList("Thor", "Athena", "Dana", "Vrinda")
ref name = list[2]
print(name)
name += "?"
print(list[2])
@@ -0,0 +1,27 @@
var name_copy = list[2] # owned copy of list[2]
ref name_ref = list[2] # reference to list[2]
```
#### Parametric mutability of return values
Another advantage of `ref` return arguments is the ability to support parametric
mutability. For example, recall the signature of the `__getitem__()` method
above:
```mojo
def __getitem__(ref self, index: Int) raises -> ref[self] String:
```
Since the `origin` of the return value is tied to the origin of `self`, the
returned reference will be mutable if the method was called using a
mutable reference. The method still works if you have an immutable reference
to the `NameList`, but it returns an immutable reference:
```mojo
def pass_immutable_list(list: NameList) raises:
print(list[2])
# list[2] += "?" # Error, this list is immutable
def main() raises:
var list = NameList("Sophie", "Jack", "Diana")
pass_immutable_list(list)
@@ -0,0 +1,105 @@
ref value_ref = list[0]
```
## Argument conventions
In all programming languages, code quality and performance is heavily dependent
upon how functions treat argument values. That is, whether a value received by
a function is a unique value or a reference, and whether it's mutable or
immutable, has a series of consequences that define the readability,
performance, and safety of the language.
In Mojo, we want to provide full [value
semantics](/docs/manual/values/value-semantics) by default, which provides
consistent and predictable behavior. But as a systems programming language, we
also need to offer full control over memory optimizations, which generally
requires reference semantics. The trick is to introduce reference semantics in
a way that ensures all code is memory safe by tracking the lifetime of every
value and destroying each one at the right time (and only once). All of this is
made possible in Mojo through the use of argument conventions that ensure every
value has only one owner at a time.
An argument convention specifies whether an argument is mutable or immutable,
and whether the function owns the value. Each convention is defined by a
keyword at the beginning of an argument declaration:
- default: The function receives an **immutable reference**. This means the
function can read the original value (it's *not* a copy), but it can't
mutate (modify) it.
- `mut`: The function receives a **mutable reference**. This means the
function can read and mutate the original value (it's *not* a copy).
- `var`: The function takes **ownership** of a value. This means the function
has exclusive ownership of the argument. The caller might choose to transfer
ownership of an existing value to this function, but that's not always what
happens. The callee might receive a newly-created value, or a copy of an
existing value.
- `ref`: The function gets a reference with a parametric mutability: that is,
it follows the mutability of the referenced value.
`ref` arguments are an advanced topic, and they're described in more detail in
[Lifetimes, origins, and references](/docs/manual/values/lifetimes/).
- `out`: A special convention used for the `self` argument in
[initializers](/docs/manual/lifecycle/life/#constructor) and for
[named results](/docs/manual/functions/#named-results). An `out`
argument is uninitialized at the beginning of the function, and must be
initialized before the function returns. Although `out` arguments show up in
the argument list, they're never passed in by the caller.
- `deinit`: A special convention used in the deinitializer and consuming-move
lifecycle methods. A `deinit` argument is initialized at the beginning of the
function, and uninitialized when the function returns.
For example, this function has one argument that's a mutable
reference and one that's immutable:
```mojo
def add(mut x: Int, y: Int):
x += y
def main():
var a = 1
var b = 2
add(a, b)
print(a) # 3
```
You've probably already seen some function arguments that don't declare a
convention. By default, all arguments use the default convention of an
immutable read-only reference. In the following sections, we'll explain
each of these conventions in more detail.
### Deinitializing arguments (`deinit`)
The `deinit` convention isn't limited to `self`. You can write methods
and functions that destruct other instances:
```mojo
struct Pair:
def destroy_other(self, deinit other: Self):
# Can take from fields of `other` here
```
Like `deinit self`, the `deinit` convention in this example tells the
compiler that `other` is tagged for destruction.
Using `deinit` means `other` is logically deinitialized at the end of the
method. Because of this, it's safe to move values out of `other`, since
the instance's lifetime is guaranteed to complete.
## Immutable arguments (default)
The default convention is an immutable read-only reference. The callee
receives an immutable reference to the argument value.
For example:
```mojo
def print_list(list: List[Int]):
print(list.__str__())
def main():
var values: List[Int] = [1, 2, 3, 4]
print_list(values)
@@ -0,0 +1,11 @@
def print_list(list: List[Int]):
print(list.__str__())
def mutate(mut l: List[Int]):
l.append(5)
def main():
var values: List[Int] = [1, 2, 3, 4]
mutate(values)
print_list(values)
@@ -0,0 +1,13 @@
def print_list(list: List[Int]):
print(list.__str__())
def mutate_copy(l: List[Int]) -> List[Int]:
# def creates an implicit copy of the list because it's mutated
l.append(5)
return l
def main():
var values: List[Int] = [1, 2, 3, 4]
values = mutate_copy(values)
print_list(values)
@@ -0,0 +1,87 @@
def append_twice(mut s: String, other: String):
# Mojo knows 's' and 'other' can't be the same string.
s += other
s += other
def invalid_access():
var my_string = "o" # Create a run-time String value
# error: passing `my_string` mut is invalid since it's also passed
# as an immutable reference
append_twice(my_string, my_string)
print(my_string)
```
This code is confusing because the user might expect the output to be `ooo`,
but since the first addition mutates both `s` and `other`, the actual output
would be `oooo`. Enforcing exclusivity of mutable references not only prevents
coding errors, it also allows the Mojo compiler to optimize code in some cases.
One way to avoid this issue when you do need both a mutable and an immutable
reference (or need to pass the same value to two arguments) is to make a copy:
```mojo
def valid_access():
var my_string = "o" # Create a run-time String value
var other_string = my_string # Create a copy of the String value
append_twice(my_string, other_string)
print(my_string)
```
Note that argument exclusivity isn't enforced for register-passable trivial
types (like `Int` and `Bool`) as they're always passed by copy. When
passing the same value into two `Int` arguments, the callee receives two
copies of the value.
## Transfer arguments (`var` and `^`)
If you want your function to take *ownership* of a value, add the `var`
keyword before the argument name.
This convention is often combined with using the postfix `^` transfer sigil
on an argument at the call site.
When using a variable, transferring a value leaves the original variable
uninitialized. You can't use the variable after the transfer until you
assign it a new value of the original type.
### Transferring with `var`
`var` behaves differently depending on whether the caller uses the `^`
transfer sigil and whether the value conforms to `Copyable`.
The `var` keyword doesn't guarantee that the function receives *the original
value*. It guarantees only that the function receives *ownership of a
value*. That happens in one of three ways:
- **Value transfer**: The caller uses the `^` transfer sigil. This
transfers the value, leaving the original variable uninitialized. The function
argument receives ownership.
- **Copying**: Without the transfer sigil, Mojo copies the value. If the
type isn't `Copyable`, this produces a compile-time error.
- **Newly created value**: The caller passes a newly created value,
such as the result of a function call. In this case, no variable owns the
value, so ownership transfers directly to the callee. For example:
```mojo
def take(var s: String):
pass
def main():
take("A brand-new String!")
```
The following code works by making a copy of the string, because `take_text()`
uses the `var` convention, and the caller doesn't include the transfer sigil:
```mojo
def take_text(var text: String):
text += "!"
print(text)
def main():
var message = "Hello" # Create a run-time String value
take_text(message)
print(message)
@@ -0,0 +1,19 @@
def main():
var message = "Hello" # Create a run-time String value
take_text(message^)
print(message) # error: use of uninitialized value 'message'
```
This is a critical feature of Mojo's lifetime checker, because it ensures that
no two variables have ownership of the same value. To fix the error, you must
not use the `message` variable after you end its lifetime with the `^` transfer
sigil. So here is the corrected code:
```mojo
def take_text(var text: String):
text += "!"
print(text)
def main():
var message = "Hello" # Create a run-time String value
take_text(message^)
@@ -0,0 +1,7 @@
def main():
var x = 1
var y = x
y += 1
print("x:", x)
print("y:", y)
@@ -0,0 +1,11 @@
def add_two(y: Int):
# y += 2 # This would cause a compiler error because `y` is immutable
# We can instead make an explicit copy:
var z = y
z += 2
print("z:", z)
def main():
var x = 1
add_two(x)
print("x:", x)
@@ -0,0 +1,17 @@
def use_closure[func: def(Int) capturing[_] -> Int](num: Int) -> Int:
return func(num)
def create_closure():
var x = 1
@__parameter
def add(i: Int) -> Int:
return x + i
var y = use_closure[add](2)
print(y)
def main():
create_closure()