Order results witin search.php by custom meta field

I need some help with ordering the results within search.php

If I search for the keyword "XYZ", by default it returns posts that have "XYZ" within the title, which is great.

As an example, let say I get the following results:
XYZ1
XYZ2
XYZ3

However, I want to sort those 3 results by a custom meta field called "priority"

so the above should look like this
XYZ2 (has priority 10)
XYZ1 (has priority 7)
XYZ3 (has priority 4)

I tried the following:

<?php
    $args = array(
    'orderby'   => 'meta_value_num',
    'meta_key'  => 'priority',
    'order'     => 'DESC'
);
query_posts($args);
while ( have_posts() ) : the_post();
    // Results
endwhile;
?>

But what I get in return are irrelevant posts ordered by priority, for example
XYZ2 (has priority 10)
ABC3 (has priority 9)
BBA4 (has priority 8)
XYZ1 (has priority 7)
... and so on

No sure if I'm missing something.

via Chebli Mohamed