Robin.Malley
/
smr
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Add a search utility 

Add a search utility
Implement a dark theme
Add an FAQ page with some info
This commit is contained in:
Robin Malley 2020-12-28 04:01:07 +00:00
parent 8a60675236
commit 3a22c15dcd
12 changed files with 318 additions and 48 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

86
assets/faq.html Normal file
View File

@ -0,0 +1,86 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>&#x1f351;</title>
<link href="/_css/milligram.css" rel="stylesheet">
<link href="/_css/style.css" rel="stylesheet">
</head>
<body class="container">
<main class="wrapper">
<h1>FAQ</h1>
An attempt to answer frequently asked questions, and document
the detail of site features. As with all software documentation,
this is how the developer <b>thinks</b> these things work,
to find out how things actually work, see the source code.
<h2>How do a register an account?</h2>
You can go <a href="/_claim">here</a> to register an account,
although you can also post pastes and comments anonymously.
<h2>Parsers? What are those?</h2>
Parsers are the way you can style the text you post. While
this software started as a pastebin clone, the need to style
text better than pastebin.com quickly became apparent. Two
parsers are currently available:
<h3>The plain parser</h3>
<p>The goal of the plain parser is to format plain text exactly as you would expect it. It does not try to to anything fancy and tries to be faithful to the plaintext representation.
<h3>Imageboard parser</h3>
<p>The imageboard parser delivers an approximation of Infinity markup, and will be familiar to those who have used imageboards before. The imageboard parser supports the following markup:
<p>Surround text with double single-quotes(') to make text <i>italic</i>
<p>Surround text with triple single-quotes to make text <b>bold</b>
<p>Surround text with underscores(_) to make it <u>underlined</u>
<p>Surround text with double asterisks(*) to make it <span class="spoiler">spoilered</span>
<p>Surround text with tildes(~) to make it <s>strike through</s>
<p>Begin a line with a greater-than followed by a space to make it
<p><span class="greentext">&gt;greentext</span>
<p>Begin a line with a less-than followed by a space to make it
<p><span class="pinktext">&lt;pinktext</span>
<p>Surround text with forum-style [spoiler] and [/spoiler] tags as a second way to <span class="spoiler2">spoiler</span>
<p>Surround text with forum-style [code] and [/code] tags to make it
<pre><code>
Preformatted and monospace
</code></pre>
<p>If you have incomplete markup at the end, it shouldn't break anything, let me know if it does.
<h2>How does search work?</h2>
<p>The search utility searches for stories on the site.
At it's most simple, it searches stories based on tags,
but it can also filter stories based on: title, author,
date, and hits. In general, the syntax for search is
<code>+|-&lt;field&gt;&gt;operator&lt;&gt;value&lt;</code>
<p>The first <code>+</code> or <code>-</code> specifies
weather to include or exclude results based on this
search, the <code>&lt;field&gt;</code> specifies what
field to search for (or search based on tag if this is
missing), and <code>&lt;operator&gt;</code> specifies
how to search.
<p>For title, and author, the only allowed operator is
<code>=</code>. This operator will search for the text
appearing anywhere in the field, case insensitive. For
hits and time, the allowed operators are
<code>&gt;</code>,<code>&lt;</code>,<code>&gt;=</code>,
<code>&lt;=</code>,<code>=</code>, which searches for
greater than, less than, greater than or equal to, less
than or equal to, and strictly equal to, respectively.
Tag does not need a field or operator, and only allows
exact matches. As a quirk of this system, it is
impossible to search for the tags "author", "title",
"hits" or "date",
Examples:
<pre><code>+author=admin -meta</code></pre>
Will return all stories by the users "admin" and "badminton_enthusiast" that do not include the "meta" tag.
<pre><code>+hits&gt;20 -date&lt;=1609459201</code></pre>
Will return all stories with more than 20 hits that were posted before January 1, 2021 (unix timestamp 1609459201).<br/>
While the date field is a little hard to use for humans, it may be useful for robots.
<h2>How do I enable reader mode on my story?</h2>
Unfortunately, there is no web standard about how reader
modes for different browsers work, and no way to hint to
the browser that a page is readable. The site makes a
best effort to trigger browsers into thinking a post is
readable, but this is unreliable. In general, if reader mode
is broken you just need to make your post a little longer.
</main>
</body>
</html>

14
assets/style.css
View File

@ -20,7 +20,19 @@ p,.tag-list{margin-bottom:0px}
padding: 0 1em 0 1em; padding: 0 1em 0 1em;
margin: 0 1px 0 1px; margin: 0 1px 0 1px;
} }
.search{
display:flex !important;
flex:0.5 0.5 auto !important;
}
.search>.button{
flex:10 10 auto;
translate: -100%;
}
.column-0{margin-right:5px;}
@media (prefers-color-scheme: dark){ @media (prefers-color-scheme: dark){
@import "css/style_dark.css"; body, form>*{
background: #1c1428;
color: #d0d4d8 !important;
}
} }

3
conf/smr.conf
View File

@ -36,6 +36,7 @@ domain * {
route /_css/milligram.css asset_serve_milligram_css route /_css/milligram.css asset_serve_milligram_css
route /_css/milligram.min.css.map asset_serve_milligram_min_css_map route /_css/milligram.min.css.map asset_serve_milligram_min_css_map
route /_css/style_dark.css asset_serve_style_dark_css route /_css/style_dark.css asset_serve_style_dark_css
route /_faq asset_serve_faq_html
route /favicon.ico asset_serve_favicon_ico route /favicon.ico asset_serve_favicon_ico
route /_paste post_story route /_paste post_story
route /_edit edit_story route /_edit edit_story
@ -77,7 +78,7 @@ domain * {
validate tags v_any validate tags v_any
} }
params get /_search { params get /_search {
validate tag v_any validate q v_any
} }
params get ^/[^_].* { params get ^/[^_].* {
validate comments v_bool validate comments v_bool

2
src/lua/endpoints/download_get.lua
View File

@ -25,7 +25,7 @@ local function download_get(req)
local err = util.do_sql(stmnt_download) local err = util.do_sql(stmnt_download)
if err == sql.DONE then if err == sql.DONE then
--No rows, story not found --No rows, story not found
http_responose(req,404,pages.nostory{path=story}) http_response(req,404,pages.nostory{path=story})
stmnt_download:reset() stmnt_download:reset()
return return
end end

4
src/lua/endpoints/index_get.lua
View File

@ -52,21 +52,17 @@ local function get_author_home(req)
stmnt_author_bio:bind_names{author=subdomain} stmnt_author_bio:bind_names{author=subdomain}
local err = util.do_sql(stmnt_author_bio) local err = util.do_sql(stmnt_author_bio)
if err == sql.DONE then if err == sql.DONE then
print("No such author")
stmnt_author_bio:reset() stmnt_author_bio:reset()
return pages.noauthor{ return pages.noauthor{
author = subdomain author = subdomain
} }
end end
print("err:",err)
assert(err == sql.ROW,"failed to get author:" .. subdomain .. " error:" .. tostring(err)) assert(err == sql.ROW,"failed to get author:" .. subdomain .. " error:" .. tostring(err))
local data = stmnt_author_bio:get_values() local data = stmnt_author_bio:get_values()
local bio = data[1] local bio = data[1]
stmnt_author_bio:reset() stmnt_author_bio:reset()
print("Getting author's stories")
stmnt_author:bind_names{author=subdomain} stmnt_author:bind_names{author=subdomain}
err = util.do_sql(stmnt_author) err = util.do_sql(stmnt_author)
print("err:",err)
local stories = {} local stories = {}
while err == sql.ROW do while err == sql.ROW do
local data = stmnt_author:get_values() local data = stmnt_author:get_values()

67
src/lua/endpoints/search_get.lua
View File

@ -6,6 +6,7 @@ local util = require("util")
local libtags = require("tags") local libtags = require("tags")
local pages = require("pages") local pages = require("pages")
local config = require("config") local config = require("config")
local search_parser = require("parser_search")
local stmnt_search local stmnt_search
local oldconfigure = configure local oldconfigure = configure
@ -18,42 +19,40 @@ local function search_get(req)
local host = http_request_get_host(req) local host = http_request_get_host(req)
local path = http_request_get_path(req) local path = http_request_get_path(req)
http_request_populate_qs(req) http_request_populate_qs(req)
local tag = http_argument_get_string(req,"tag") local searchq = assert(http_argument_get_string(req,"q"))
if tag then log(LOG_DEBUG,string.format("search: %q",searchq))
stmnt_search:bind_names{ local sqltxt, data = search_parser(searchq)
tag = tag local stmnt = assert(db.conn:prepare(sqltxt), db.conn:errmsg())
} local i = 1
local results = {} for field,values in pairs(data) do
local err if field ~= "tags" then
repeat for _, value in pairs(values) do
err = stmnt_search:step() stmnt:bind(i,value[3])
if err == sql.BUSY then i = i + 1
coroutine.yield()
elseif err == sql.ROW then
local id, title, anon, time, author = unpack(stmnt_search:get_values())
local idp = util.encode_id(id)
local tags = libtags.get(id)
table.insert(results,{
id = idp,
title = title,
anon = anon,
time = os.date("%B %d %Y",tonumber(time)),
author = author,
tags = tags
})
elseif err == sql.DONE then
stmnt_search:reset()
else
error("Failed to search, sql error:" .. tostring(err))
end end
until err == sql.DONE end
local ret = pages.search{
domain = config.domain,
results = results,
tag = tag,
}
http_response(req,200,ret)
end end
for _,value in pairs(data.tags) do
stmnt:bind(i,value[3])
i = i + 1
end
local results = {}
for row in stmnt:rows() do
table.insert(results,{
id = util.encode_id(row[1]),
title = row[2],
isanon = row[3] == 1,
author = row[4],
time = os.date("%B %d %Y",tonumber(row[5])),
tags = libtags.get(row[1])
})
end
local ret = pages.search{
domain = config.domain,
results = results,
q = searchq,
}
http_response(req,200,ret)
end end
return search_get return search_get

78
src/lua/parser_search.lua
View File

@ -1,10 +1,84 @@
local lpeg = require('lpeg')
local etlua = require('etlua')
local args = {...}
lpeg.locale(lpeg)
local V,P,C,S,B,Cs = lpeg.V,lpeg.P,lpeg.C,lpeg.S,lpeg.B,lpeg.Cs
--Identity function
local ident = function(a) return a end
--Lowercase and capitalize first letter
local function capitalize(str)
return string.lower(str):gsub("^(.)",string.upper)
end
--Trim whitespace
local function trim(str)
return str:match("^%s*(.-)%s*$")
end
--SQL like match anywhere
local function like(str)
return "%" .. str .. "%"
end
--Tags are always trimed of whitepsace, lowercase with first letter capitalized
local function tag_fmt(str)
return capitalize(trim(str))
end
--Title match
local function title_fmt(str)
return like(trim(str))
end
--Author names are all lowercase alphanumeric, max 30 characters
local function author_fmt(str)
return like(string.lower(trim(str)))
end
local fieldnames = {
title = {name="title",type="string",fmt=title_fmt},
author = {name="author",type="string",fmt=author_fmt},
date = {name="date",type="number",fmt=tonumber},
hits = {name="hits",type="number",fmt=tonumber},
tags = {name="tag",type="string",fmt=tag_fmt},
}
local field_default = "tag"
local fields
local grammar = P{
"chunk";
whitespace = S" \t\n"^0,
itm = C(P(1-S"+-")^0), --go until the next '+' or '-'
likefield = C(P"title" + P"author") * V"whitespace" * C(P"=") * V"whitespace" * V"itm",
rangeop = P"<=" + P">=" + P">" + P"<" + P"=",
rangefield = C(P"date" + P"hits") * V"whitespace" * C(V"rangeop") * V"whitespace" * C(V"itm"),
field = C(S"+-") * (V"likefield" + V"rangefield" + V"itm") / function(pn,field,expr,value)
if expr and value then
fields[field] = fields[field] or {}
table.insert(fields[field],{pn,expr,value})
else
fields.tags = fields.tags or {}
table.insert(fields.tags,{pn,"=",field})
end
end,
chunk = V"field"^0
}
--Grammar --Grammar
--Transpile a sting with + and - into an sql query that searches tags --Transpile a sting with + and - into an sql query that searches tags
local fname = "pages/search_sql.etlua"
local sqltmpl = assert(io.open(fname))
local c = etlua.compile(sqltmpl:read("*a"),fname)
sqltmpl:close()
local function transpile(str) local function transpile(str)
for chunk in str:gmatch("([+-])([^+-])") do str = string.lower(str)
print("found chunk:",chunk) fields = {tags={}}
table.concat({grammar:match(str)}," ")
--Sanity perform formatting on data
for field,values in pairs(fields) do
for _,value in pairs(values) do
local pn, expr, val = unpack(value)
local nval = fieldnames[field].fmt(val)
value[3] = nval
end
end end
local ressql = c{
result = fields
}
return ressql, fields
end end
return transpile return transpile

44
src/lua/search_sql.etlua Normal file
View File

@ -0,0 +1,44 @@
SELECT
posts.id,
posts.post_title,
authors.name,
posts.post_time
FROM
posts,authors
WHERE
authors.id = posts.authorid
<% print("reqult:",result) -%>
<% for field, values in pairs(result) do -%>
<% print("values:",values) -%>
<% for _,value in pairs(values) do -%>
<% local pn,expr,value = unpack(value) -%>
<% local n = (pn == "+" and "" or "NOT") -%>
<% if field == "title" then -%>
AND <%= n %> posts.post_title LIKE ?
<% elseif field == "author" then -%>
AND <%= n %> authors.name LIKE ?
<% elseif field == "date" then -%>
AND <%= n %> posts.post_time <%- expr %> ?
<% elseif field == "hits" then -%>
AND posts.views <%- expr -%> ?
<% end -%>
<% end -%>
<% end -%>
<% print("tags:",result.tags) %>
<% for _,tag in pairs(result.tags) do -%>
INTERSECT
SELECT
posts.id,
posts.post_title,
authors.name,
posts.post_time
FROM
posts,authors,tags
WHERE
posts.authorid = authors.id
AND tags.postid = posts.id
<% local n,v = unpack(tag) -%>
<% n = (n == "-" and "NOT" or "") -%>
AND <%= n %> tags.tag = <%= v %>
<% end -%>
;

2
src/lua/util.lua
View File

@ -45,7 +45,7 @@ function util.sqlbind(stmnt,call,position,data)
assert(call == "bind" or call == "bind_blob","Bad bind call, call was:" .. call) assert(call == "bind" or call == "bind_blob","Bad bind call, call was:" .. call)
local f = stmnt[call](stmnt,position,data) local f = stmnt[call](stmnt,position,data)
if f ~= sql.OK then if f ~= sql.OK then
error(string.format("Failed to %s at %d with %q: %s", call, position, data, db:errmsg()),2) error(string.format("Failed to %s at %d with %q: %s", call, position, data, db.conn:errmsg()),2)
end end
end end

14
src/pages/index.etlua
View File

@ -15,9 +15,15 @@
</h1> </h1>
<div class="container"> <div class="container">
<a href="/_paste" class="button">New paste</a> <div class="row">
<a href="/_login" class="button">Log in</a> <a href="/_paste" class="button column column-0">New paste</a>
<a href="/_claim" class="button">Register</a> <a href="/_login" class="button column column-0">Log in</a>
<a href="/_claim" class="button column column-0">Register</a>
<form action="https://<%= domain %>/_search" method="get" class="search column row">
<input class="column" type="text" name="q" placeholder="+greentext -dotr +hits>20"/>
<input class="column column-0 button button-clear" type="submit" value="&#x1F50E;"/>
</form>
</div>
<p> <p>
Welcome to slash.monster, stories of fiction and fantasy<br/> Welcome to slash.monster, stories of fiction and fantasy<br/>
Not safe for work<br/> Not safe for work<br/>
@ -43,7 +49,7 @@
</td><td> </td><td>
<ul class="row tag-list"> <ul class="row tag-list">
<% for i = 1,math.min(#v.tags, 5) do %> <% for i = 1,math.min(#v.tags, 5) do %>
<li><a class="tag button button-outline" href="https://<%= domain %>/_search?tag=<%= v.tags[i] %>"><%= v.tags[i] %></a></li> <li><a class="tag button button-outline" href="https://<%= domain %>/_search?q=+<%= v.tags[i] %>"><%= v.tags[i] %></a></li>
<% end %> <% end %>
<% if #v.tags > 5 then %> <% if #v.tags > 5 then %>
<li>+<%= #v.tags - 5 %></li> <li>+<%= #v.tags - 5 %></li>

9
src/pages/search.etlua
View File

@ -11,6 +11,12 @@
<h1 class="title"> <h1 class="title">
<a href="https://<%= domain %>"><%= domain %></a>/ <a href="https://<%= domain %>"><%= domain %></a>/
</h1> </h1>
<div class="row">
<form action="https://<%= domain %>/_search" method="get" class="search column row">
<input class="column" type="text" name="q" placeholder="+greentext -dotr +hits>20" value="<%= q %>"/>
<input class="column column-0 button button-clear" type="submit" value="&#x1F50E;"/>
</form>
</div>
<div class="content"> <div class="content">
<% if #results == 0 then %> <% if #results == 0 then %>
No stories matched your search. No stories matched your search.
@ -32,6 +38,9 @@
<% for i = 1,math.min(#v.tags, 5) do %> <% for i = 1,math.min(#v.tags, 5) do %>
<li><a class="tag button button-outline" href="https://<%= domain %>/_search?tag=<%= v.tags[i] %>"><%= v.tags[i] %></a></li> <li><a class="tag button button-outline" href="https://<%= domain %>/_search?tag=<%= v.tags[i] %>"><%= v.tags[i] %></a></li>
<% end %> <% end %>
<% if #v.tags > 5 then %>
<li>+<%= #v.tags - 5%></li>
<% end %>
</ul> </ul>
</td><td> </td><td>
<%= v.time %> <%= v.time %>

43
src/pages/search_sql.etlua Normal file
View File

@ -0,0 +1,43 @@
SELECT
posts.id,
posts.post_title,
posts.isanon,
authors.name,
posts.post_time
FROM
posts,authors
WHERE
authors.id = posts.authorid
<% for field, values in pairs(result) do -%>
<% for _,value in pairs(values) do -%>
<% local pn,expr,value = unpack(value) -%>
<% local n = (pn == "+" and "" or "NOT") -%>
<% if field == "title" then -%>
AND <%= n %> posts.post_title LIKE ?
<% elseif field == "author" then -%>
AND <%= n %> authors.name LIKE ?
<% elseif field == "date" then -%>
AND <%= n %> posts.post_time <%- expr %> ?
<% elseif field == "hits" then -%>
AND posts.views <%- expr -%> ?
<% end -%>
<% end -%>
<% end -%>
<% for _,tag in pairs(result.tags) do -%>
INTERSECT
SELECT
posts.id,
posts.post_title,
posts.isanon,
authors.name,
posts.post_time
FROM
posts,authors,tags
WHERE
posts.authorid = authors.id
AND tags.postid = posts.id
<% local n,v,t = unpack(tag) -%>
<% n = (n == "-" and "NOT" or "") -%>
AND <%= n %> tags.tag = ?
<% end -%>
;