[Wayland-bugs] [Bug 93315] Automatically scale windows?

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

Bug ID: 93315
Summary: Automatically scale windows?
Product: Wayland
Version: unspecified
Hardware: Other
OS: All
Status: NEW
Severity: normal
Priority: medium
Component: XWayland
Assignee: wayland-***@lists.freedesktop.org
Reporter: ***@andreasn.se
QA Contact: xorg-***@lists.x.org

I was talking to Alex about this in the office and he suggested filing a bug
about it, so here we go.

Currently on a hidpi-screen, when running GTK3 apps, such as gedit, totem etc.
everything works fine, since the toolkit supports hidpi.
However, older applications, such as Inkscape and GIMP that uses GTK2 becomes
super-tiny on the same device. Waiting for them to move over to a newer toolkit
version is of course nice, but perhaps wayland could scale those that aren't
yet up, since blurry graphics is to prefer over graphics that is too tiny to
see.

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #1 from Alexander Larsson <***@redhat.com> ---
Xwayland should set the scale of the buffers to 1, and then the compositor
should automatically scale it up if the output has scale 2. But instead when I
run a X11 app it seems to pick larger fonts, but render everything else tiny.

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #2 from Pekka Paalanen <***@gmail.com> ---
Have you checked that the Wayland compositor actually is configured to have
output scale 2? Or that it is not deliberately showing X11 windows unscaled
because some X11 toolkits might do hidpi in secret and you don't want to scale
up twice?

If X11 apps cannot indicate in which scale they render, then I'd assume
Xwayland has no other choice than to use buffer scale 1. When the Wayland
compositor is configured correctly, it will scale up all X11 windows. The
downside is that Xwayland cannot support hidpi apps. So, you'd have to somehow
get that scale info from X11 apps to have them not scaled up twice. Is there
any X11 protocol for that?

I gave a quick try with Weston and xterm, and output scale 2 causes the xterm
to be scaled up like you would expect.

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #3 from Alexander Larsson <***@redhat.com> ---
Unfortunately there is no X APIs to get the scale, each toolkit makes some
guess on their own.

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #4 from Pekka Paalanen <***@gmail.com> ---
Right, so either a Wayland compositor scales all X11 windows or none. If you
scale none, you miss non-hidpi-aware X11 apps. If you scale all, you have to
somehow ensure hidpi-aware X11 apps do not scale themselves.

I suppose there is nothing to do in Xwayland for this? Or should it be lying
something to avoid X11 apps scaling themselves?

Should this bug target some other piece of software? Like a compositor that
doesn't scale X11 windows?

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #5 from Daniel Stone <***@fooishbar.org> ---
Server-side decorations also make it really hard, since you need to make sure
the decorations are rendered at the appropriate scale.

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #6 from Alexander Larsson <***@redhat.com> ---
I've not looked at the gnome-shell code, but running wayland gnome-shell with
scale=2 i get X11 windows that have tiny icons but normal text, so i assume it
reports the real pixel size to the X11 apps and a large dpi.

It would be nicer if it returned a smaller resolution and a lower dpi, unless
the app somehow opted in to supporting hidpi.

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #7 from Daniel Stone <***@fooishbar.org> ---
(In reply to Alexander Larsson from comment #6)

Post by b***@freedesktop.org
I've not looked at the gnome-shell code, but running wayland gnome-shell
with scale=2 i get X11 windows that have tiny icons but normal text, so i
assume it reports the real pixel size to the X11 apps and a large dpi.
It would be nicer if it returned a smaller resolution and a lower dpi,
unless the app somehow opted in to supporting hidpi.

It would be nice, but unfortunately X11 has the global co-ordinate space very
deeply baked in to its entire API. There's no good way to do this.

I think the only solution will be to work out a way for apps to declare that
they are indeed HiDPI-aware and have scaled appropriately (e.g.
_NET_WM_SCALE_FACTOR == 2), have the server watch those, and pass that through
to the host compositor as appropriate, and make sure that whatever's rendering
decorations accounts for this.

Except then you run into the very difficult question of window placement: you'd
need to offset all transients by the appropriate amount as well. So, if you
have a window which you're scaling up by 2x, and it wants to create a transient
at an offset of (40,40) relative to the primary, you need to shift that to
(80,80) accordingly.

The window manager would also need to enforce this wrt window configuration: if
you have a 3200x1800 output, it needs to clamp maximisation to 1600x900, and so
on, and so forth.

Not particularly pretty ...

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #8 from Alexander Larsson <***@redhat.com> ---
Could we run two xwayland instances? first display is lodpi, and second is
hidpi? Then we have some magic desktop file thing to tell the shell to launch
hidpi apps on display 2?

[b***@freedesktop.org]

Dette indlæg kan være upassende. Klik for at få det vist.

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #10 from Olivier Fourdan <***@xfce.org> ---
(In reply to Daniel Stone from comment #9)

Post by b***@freedesktop.org
(In reply to Alexander Larsson from comment #8)

Post by b***@freedesktop.org
Could we run two xwayland instances? first display is lodpi, and second is
hidpi? Then we have some magic desktop file thing to tell the shell to
launch hidpi apps on display 2?

Not without some pretty radical surgery, no. Mutter would need a huge rework
to be a WM across multiple X servers at the same time, as would the XWayland
integration (e.g. to cope with XIDs no longer being unique).

Not sure if that'd be workable, but What about 2 screens of the same display
then (:0.0 and :0.1)? Each could have its own resolution/size/DPI, and yet map
to the same Wayland output.

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #11 from Daniel Stone <***@fooishbar.org> ---
(In reply to Olivier Fourdan from comment #10)

Post by b***@freedesktop.org
Not sure if that'd be workable, but What about 2 screens of the same display
then (:0.0 and :0.1)? Each could have its own resolution/size/DPI, and yet
map to the same Wayland output.

Mm, a much better idea, but still some pretty radical surgery. Screens aren't
hotpluggable for starters (IIRC), and also they're the root of Drawables, so
you can't blit Pixmaps across Screens, have a Window spanning Screens, or move
a Window between Screens. :(

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #12 from Alexander Larsson <***@redhat.com> ---
I don't think that is really a problem though.

not dynamic: We could just always allocate a scale 2 screen.

can't blit across screens: When would you need that. Apps would generally just
connect to either one, and ignore the other. The compositor can do blitting on
the wayland side.

windows can't share screens or move to other screen: Well, either the app has
no knowledge of hidpi, we then always want it to be on the screen that has
scale=1 (i.e. is scaled up if the real output is scale=2). Apps that are hidpi
aware are passed the unmodified screen (and some env var is set about the
scale), and then they can render in full resolution. Then the compositor can
scale these down to scale=1 outputs in the mixed-scale case.

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #13 from Alexander Larsson <***@redhat.com> ---
s/env var/root window x-property/

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #14 from Daniel Stone <***@fooishbar.org> ---
(In reply to Alexander Larsson from comment #12)

Post by b***@freedesktop.org
I don't think that is really a problem though.
not dynamic: We could just always allocate a scale 2 screen.
can't blit across screens: When would you need that. Apps would generally
just connect to either one, and ignore the other. The compositor can do
blitting on the wayland side.
windows can't share screens or move to other screen: Well, either the app
has no knowledge of hidpi, we then always want it to be on the screen that
has scale=1 (i.e. is scaled up if the real output is scale=2). Apps that are
hidpi aware are passed the unmodified screen (and some env var is set about
the scale), and then they can render in full resolution. Then the compositor
can scale these down to scale=1 outputs in the mixed-scale case.

Hm, right; I think that could well work, but would require a fair bit of
surgery. Especially on the XWayland side, where we have to perform the mapping
back into a fixed/linear co-ordinate space. But yeah, it could well just work
...

[b***@freedesktop.org]

Dette indlæg kan være upassende. Klik for at få det vist.

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #16 from Adam Goode <***@spicenitz.org> ---
Will these ideas also work for fractional scales? 1.25x screens are common now.

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #17 from Olivier Fourdan <***@xfce.org> ---
(In reply to Adam Goode from comment #16)

Post by b***@freedesktop.org
Will these ideas also work for fractional scales? 1.25x screens are common
now.

The scaling facter is an integer in Wayland protocol:

<event name="scale" since="2">
<description summary="output scaling properties">
This event contains scaling geometry information
that is not in the geometry event. It may be sent after
binding the output object or if the output scale changes
later. If it is not sent, the client should assume a
scale of 1.

A scale larger than 1 means that the compositor will
automatically scale surface buffers by this amount
when rendering. This is used for very high resolution
displays where applications rendering at the native
resolution would be too small to be legible.

It is intended that scaling aware clients track the
current output of a surface, and if it is on a scaled
output it should use wl_surface.set_buffer_scale with
the scale of the output. That way the compositor can
avoid scaling the surface, and the client can supply
a higher detail image.
</description>
<arg name="factor" type="int" summary="scaling factor of output"/>
</event>

Beside, I am not sure how a quarter of a pixel would look like.

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #18 from Alexander Larsson <***@redhat.com> ---
For wayland the plan is for the app to submit scale=2 buffers and for
coordinates (window pos, cursor pos, etc) to be scale=2, but the compositor
downscales to 1.5 in the final composite to the scanout buffer.

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #19 from Shane Chen <***@gmail.com> ---
(In reply to Alexander Larsson from comment #18)

Post by b***@freedesktop.org
For wayland the plan is for the app to submit scale=2 buffers and for
coordinates (window pos, cursor pos, etc) to be scale=2, but the compositor
downscales to 1.5 in the final composite to the scanout buffer.

I'm curious about how to downscale to 1.5.
As the wl_ouput::scale can only be int , so if the compositor is rendering in
1.5 scale, it will tell the client that the output scale is 2? So the client
will never aware about the actual output scale 1.5?

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #20 from Alexander Larsson <***@redhat.com> ---
Yeah, which is how e.g. MacOS X handles this.

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

Olivier Fourdan <***@xfce.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
See Also| |https://bugs.freedesktop.or
| |g/show_bug.cgi?id=101193

[b***@freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=93315

--- Comment #21 from Pekka Paalanen <***@gmail.com> ---
A new interesting idea for window scaling from Keith:
https://lists.x.org/archives/xorg-devel/2018-August/057474.html